An automated system is proposed for the detection and classification of GI abnormalities. The proposed method operates under two pipeline procedures: (a) segmentation of the bleeding infection region and (b) classification of GI abnormalities by deep learning. The first bleeding region is segmented using a hybrid approach. The threshold is applied to each channel extracted from the original RGB image. Later, all channels are merged through mutual information and pixel-based techniques. As a result, the image is segmented. Texture and deep learning features are extracted in the proposed classification task. The transfer learning (TL) approach is used for the extraction of deep features. The Local Binary Pattern (LBP) method is used for texture features. Later, an entropy-based feature selection approach is implemented to select the best features of both deep learning and texture vectors. The selected optimal features are combined with a serial-based technique and the resulting vector is fed to the Ensemble Learning Classifier. The experimental process is evaluated on the basis of two datasets: Private and KVASIR. The accuracy achieved is 99.8 per cent for the private data set and 86.4 percent for the KVASIR data set. It can be confirmed that the proposed method is effective in detecting and classifying GI abnormalities and exceeds other methods of comparison.
Computerized detection of human diseases is an emerging research domain for the last two decades [
GI infections can be easily cured if they were diagnosed at an early stage. As small bowel has a complex structure, that is why push gastroscopy is not considered as the best choice for the diagnosis of small bowel infections like bleeding, polyp, and ulcer [
In this work, to detect and classify GI abnormalities, two pipeline procedures are considered: bleeding segmentation and GI abnormality classification. The significant contributions of the work are: i) in the bleeding detection step, a hybrid technique is implemented. In this method, the original image is split into three channels and thresholding is applied for each color channel. After that, pixel-by-pixel matching is performed, and a mask is generated for each channel. Finally, combining all mask images in one image for final bleeding segmentation; ii) in the classification procedure, transfer learning is ustilized for extracting deep learning features. The original images are enhanced by using a unified method, which is a combination of chrominance weight map, gamma correction, haze reduction, and YCbCr color conversion. Later, deep learning features are extracted by using a pre-trained CNN model. Further, the LBP features are also obtained for textural information of each abnormal regions; iii) a new method named entropy controlled ensemble learning is proposed and it selects the best learning features for correct prediction as well as fast execution. The selected features are ensemble in one vector by using a concatenation approach; iv) the performance of the proposed method is validated by several combinations of features. Further, many classification methods are also used for validation of selected features vector.
Several machine learning and computer vision-based techniques are introduced for the diagnosis of human diseases like lung cancer, brain tumor, GIT infections from WCE images, and so on [
In contrast, the images that have the same category should share the same learned features. The overall achieved recognition accuracy of this method is 98%. Sharif et al. [
A hybrid architecture is proposed in this work for automated detection and classification of stomach abnormalities. The proposed architecture follows two pipeline procedures: Bleeding abnormality segmentation and GI infections classification. The proposed bleeding segmentation procedure is illustrated in
Consider
Considering three channels
where,
where,
Bi-class variance can be expressed as:
In Ostu segmentation, threshold selection is based on the cost function and can be calculated as:
where,
Suppose,
where
In the classification task, we presented a deep learning-based architecture for GI abnormalities classification such as bleeding, ulcer, and healthy. This task consists of three steps: enhancement of original images, deep feature extraction, and selection of robust features for classification. The flow diagram is presented in
WCE images may suffer from non-uniform lighting, low visibility, diminished colors, blurring, and low contrast of image characteristics [
where,
where,
The foundation of a hazy image can be presented as a curved combination of the image radiance
Using this model, the ratio that we have used for training and testing is 70:30. This feature extraction process is conducted using the transfer learning method. In the TL, a pre-trained model was trained on GI WCE images. For this purpose, we required an input layer and an output layer. For input layer, we are using the first convolution layer, while in the output; we select the average pool layer. We have obtained a feature vector of dimension
Here, the number of neighborhood intensities are represented by
where neighboring pixels
After the extraction of texture and deep learning features, the next phase involves the optimal features selection. In this work, we have utilized Shannon entropy along with an ensemble learning classifier for best features selection. A heuristic approach has opted for feature selection. The Shannon Entropy is computed from both vectors separately and set a target function based on the mean value of original entropy vectors. The features that are equal or higher than mean features are selects as robust features and passed to ensemble classifiers. However, this process is to continue until the error rate of the ensemble classifier is below 0.1. Mathematically Shannon entropy is ratified by the equation as follow:
Let
The Shannon entropy
Through this process, approximately 50% of features are removed from both vectors-deep learning and texture oriented. Later on, these selected vectors are fused in one vector by simple concatenation approach as given as: Let
Two datasets are used in this work for the assessment of suggested GI infections detection and classification method. The description of each dataset is given as:
A detailed description of classification results in quantitative and graphical form is given in this section. For experimental results, seven different classifiers are used for the evaluation of suggested methods that are Linear Discriminant (L-Disc), Fine Tree (F-Tree), Cubic SVM (C-SVM), Medium Gaussian SVM (M-G-SVM), Linear SVM (L-SVM), Fine KNN (F-KNN), and ENSEM subspace discriminant (ESDA). In this research, we have utilized different performance measures for the evaluation of suggested methods, including Specificity (SPE), FNR, Precision (PRE), FPR, Sensitivity (SEN), Accuracy (ACC), F1-Score, Jaccard index, and Dice. All the tests are implemented on MATLAB 2019b using Core i5-7thGen, 4 GB RAM. Further, an 8 GB graphics card is also used for the evaluation of results.
Image # | Performance measures | |||
---|---|---|---|---|
Jack-index (%) | Dice (%) | FNR (%) | Time (s) | |
1 | 95.32 | 91.05 | 8.95 | 5.865451 |
2 | 93.74 | 88.21 | 11.79 | 6.481893 |
3 | 88.25 | 78.98 | 21.02 | 6.109725 |
4 | 91.46 | 84.27 | 15.73 | 14.66179 |
5 | 95.02 | 90.51 | 9.49 | 5.103764 |
6 | 96.23 | 92.73 | 7.27 | 8.664483 |
7 | ||||
8 | 90.81 | 83.17 | 16.83 | 6.732851 |
9 | 94.34 | 89.29 | 5.66 | 9.608909 |
10 | 88.51 | 79.38 | 20.62 | 9.828089 |
11 | 93.66 | 88.08 | 11.92 | 9.818192 |
12 | 94.98 | 90.44 | 9.56 | 10.99683 |
13 | 93.62 | 88.01 | 11.99 | 8.654121 |
14 | 96.23 | 92.74 | 7.26 | 17.56766 |
15 | 90.38 | 82.44 | 17.56 | 13.99112 |
16 | 95.63 | 91.62 | 8.38 | 13.67921 |
17 | 93.46 | 87.72 | 12.28 | 7.435595 |
18 | 94.71 | 89.94 | 10.06 | 11.13178 |
19 | 95.25 | 90.94 | 9.06 | 8.380193 |
20 | 88.25 | 78.98 | 11.75 | 5.927833 |
Classifier | Performance measures | |||||||
---|---|---|---|---|---|---|---|---|
ACC (%) | SEN (%) | SPE (%) | PRE (%) | F1 score (%) | FPR | FNR (%) | Execution time (s) | |
F-Tree | 88.1 | 86.83 | 94.19 | 86.32 | 86.55 | 0.06 | 11.9 | 22.82 |
L-Disc | 99.4 | 99.30 | 99.68 | 99.39 | 99.34 | 0.00 | 0.6 | 25.88 |
L-SVM | 98.8 | 98.57 | 99.34 | 98.86 | 98.72 | 0.01 | 1.2 | 18.14 |
C-SVM | 99.4 | 99.30 | 99.68 | 99.39 | 99.34 | 0.00 | 0.6 | 18.01 |
M-G-SVM | 99.5 | 99.39 | 99.70 | 99.55 | 99.47 | 0.00 | 0.5 | 26.16 |
F-KNN | 99.4 | 99.28 | 99.72 | 99.39 | 99.33 | 0.00 | 0.6 | 111.2 |
ESDA | 0.00 |
Classes | Classification classes | ||
---|---|---|---|
Bleeding | Healthy | Ulcer | |
Bleeding | < 1% | ||
Healthy | < 1% | ||
Ulcer | 1% |
Classifier | Performance measures | |||||||
---|---|---|---|---|---|---|---|---|
ACC (%) | SEN (%) | SPE (%) | PRE (%) | F1 Score (%) | FPR | FNR (%) | Execution time (s) | |
F-Tree | 56.3 | 56.33 | 93.76 | 56.22 | 56.24 | 0.06 | 43.7 | |
L-Disc | 84.9 | 84.93 | 97.85 | 85.26 | 85.05 | 0.02 | 15.1 | 27.84 |
L-SVM | 85.7 | 85.65 | 97.95 | 85.63 | 85.59 | 0.02 | 14.3 | 46.46 |
C-SVM | 85.8 | 85.23 | 97.93 | 85.23 | 85.22 | 0.02 | 14.2 | 53.72 |
M-G-SVM | 84.7 | 84.70 | 97.81 | 84.82 | 84.67 | 0.02 | 15.3 | 29.36 |
F-KNN | 74.9 | 72.18 | 96.03 | 73.09 | 72.12 | 0.04 | 25.1 | 50.30 |
ESDA | 0.02 | 43.01 |
Classes | Classification classes | |||||||
---|---|---|---|---|---|---|---|---|
DLP | DRM | ESO | NC | NP | NZL | P | UCE | |
DLP | 19% | 3% | 1% | |||||
DRM | 20% | < 1% | 1% | < 1% | ||||
ESO | 1% | 22% | < 1% | < 1% | ||||
NC | 2% | 1% | ||||||
NP | 1% | 1% | ||||||
NZL | 13% | 1% | 1% | |||||
P | 1% | 4% | 1% | < 1% | 5% | |||
UCE | 1% | 1% | 1% | 3% |
Later on, the selected deep learning and LBP features are fused and performed evaluation. Results are presented in
Classifier | Performance measures | |||||||
---|---|---|---|---|---|---|---|---|
ACC (%) | SEN (%) | SPE (%) | PRE (%) | F1 score (%) | FPR | FNR (%) | Execution time (s) | |
F-Tree | 87.7 | 86.23 | 94.02 | 85.85 | 86.03 | 0.06 | 12.3 | 27.373 |
L-Disc | 99.5 | 99.47 | 99.75 | 99.52 | 99.49 | 0.00 | 0.5 | 19.944 |
L-SVM | 98.8 | 98.65 | 99.36 | 98.89 | 98.77 | 0.01 | 1.2 | 20.426 |
C-SVM | 99.6 | 99.65 | 99.69 | 99.67 | 99.66 | 0.00 | 18.217 | |
M-G-SVM | 99.4 | 99.34 | 99.68 | 99.52 | 99.43 | 0.00 | 0.6 | 25.439 |
F-KNN | 99.6 | 99.50 | 99.81 | 99.58 | 99.54 | 0.00 | 0.4 | 42.39 |
ESDA | 0.00 |
The fused vector is also applied to the KVASIR dataset and achieved maximum accuracy of 87.8% for the ESDA classifier.
Classes | Classification classes | ||
---|---|---|---|
Bleeding | Healthy | Ulcer | |
Bleeding | |||
Healthy | ¡1% | ||
Ulcer | < 1% |
Classifier | Performance measures | |||||||
---|---|---|---|---|---|---|---|---|
ACC (%) | SEN (%) | SPE (%) | PRE (%) | F1 score (%) | FPR | FNR (%) | Execution time (s) | |
F-Tree | 60.4 | 60.40 | 94.34 | 60.29 | 60.34 | 0.06 | 39.6 | 72.29 |
L-Disc | 84.8 | 84.80 | 97.83 | 85.23 | 84.98 | 0.02 | 15.2 | 32.34 |
L-SVM | 86.1 | 86.05 | 98.01 | 86.19 | 86.05 | 0.02 | 13.9 | 94.90 |
C-SVM | 85.7 | 85.73 | 97.96 | 85.79 | 85.73 | 0.02 | 14.3 | 79.03 |
M-G-SVM | 84.6 | 84.60 | 97.80 | 84.82 | 84.64 | 0.02 | 15.4 | 46.95 |
F-KNN | 75.3 | 72.83 | 96.12 | 73.8 | 72.83 | 0.04 | 24.7 | |
ESDA | 0.02 | 50.70 |
Classes | Classification classes | |||||||
---|---|---|---|---|---|---|---|---|
DLP | DRM | ESO | NC | NP | NZL | P | UCE | |
19% | < 1% | 2% | ||||||
21% | 1% | < 1% | ||||||
1% | 21% | |||||||
3% | 1% | |||||||
1% | 1% | |||||||
12% | 1% | ¡1% | ||||||
1% | 4% | 2% | 1% | 5% | ||||
1% | 1% | 1% | 3% |
From the above review, it is noted that several methods are proposed by the researchers for GI disease abnormality detection and classification. Most of the existing CAD methods are based on traditional techniques and algorithms, such as most of them are based on only color information or texture information. Although there exists some methods in which authors have used a combination of features. Some methods are also based on deep CNN features. Despite too many existing CAD methods, there exist some limitations in the old approaches such as low contrast of captured frames, the same color of the infected and healthy region, the problem of proper color model selection, hazy image, redundant information, etc. These limitations forced us to develop a robust method for GI abnormality detection and classification with better accuracy. The proposed deep learning method is evaluated on two datasets-Kvasir and Private and achieved an accuracy of 99.80% and 87.80%. The comparison with existing techniques is given in
Ref. | Year | Disease/dataset | Performance measures | ||||
---|---|---|---|---|---|---|---|
ACC (%) | SEN (%) | SPE (%) | PRE (%) | Execution time (s) | |||
[ |
2019 | Bleeding, Ulcer and Normal | 99.54 | 100 | – | 99.51 | 12.59 |
[ |
2019 | Ulcer, Cancer, Normal | 96.49 | ||||
[ |
2018 | Ulcer, bleeding | 98.49 | 98.00 | 98.00 | 99.00 | 17.393 |
Private dataset: Bleeding, Ulcer and Normal | |||||||
KVASIR dataset |
In this article, we proposed a deep learning architecture for the detection and classification of GI abnormalities. The proposed architecture consists of two procedures for pipeline detection and classification. In the detection task, the bleeding region is segmented by a fusion of three separate channels. In the classification task, deep learning features and texture-oriented features are extracted and the best features are selected using the Shanon Entropy controlled ESDA classifier. The selected features are concatenated and are classified. In the evaluation phase, the segmentation process achieves an average accuracy of over 87% for abnormal bleeding regions. For classification, the accuracy of the private data set is 99.80 percent, while for the Kvasir data set, the accuracy is 87.80 percent. It is concluded from the results that the proposed selection method shows better performance compared to the existing techniques. It also concludes that the merger process is effective for more classes, such as the Kvasir dataset classification. In addition, texture features also have a high impact on disease classification and deep learning fusion, addressing the issue of texture variation. In future studies, we will focus on ulcer segmentation through deep learning.