In developing countries, medical diagnosis is expensive and time consuming. Hence, automatic diagnosis can be a good cheap alternative. This task can be performed with artificial intelligence tools such as deep Convolutional Neural Networks (CNNs). These tools can be used on medical images to speed up the diagnosis process and save the efforts of specialists. The deep CNNs allow direct learning from the medical images. However, the accessibility of classified data is still the largest challenge, particularly in the field of medical imaging. Transfer learning can deliver an effective and promising solution by transferring knowledge from universal object detection CNNs to medical image classification. However, because of the inhomogeneity and enormous overlap in intensity between medical images in terms of features in the diagnosis of Pneumonia and COVID-19, transfer learning is not usually a robust solution. Single-Image Super-Resolution (SISR) can facilitate learning to enhance computer vision functions, apart from enhancing perceptual image consistency. Consequently, it helps in showing the main features of images. Motivated by the challenging dilemma of Pneumonia and COVID-19 diagnosis, this paper introduces a hybrid CNN model, namely SIGTra, to generate super-resolution versions of X-ray and CT images. It depends on a Generative Adversarial Network (GAN) for the super-resolution reconstruction problem. Besides, Transfer learning with CNN (TCNN) is adopted for the classification of images. Three different categories of chest X-ray and CT images can be classified with the proposed model. A comparison study is presented between the proposed SIGTra model and the other related CNN models for COVID-19 detection in terms of precision, sensitivity, and accuracy.
Pneumonia is a disease that affects one or both lungs and aggravates air sacs. Fluid or pus (purulent material) fill the bags with air, affecting pus or phlegm to cough, fever, breathing, and chills difficulties. Pneumonia may be hard to identify because of the variability of symptoms and the occurrence of Pneumonia with cold and flu cases. Hence, Pneumonia is detected, and the germ causing the disease is determined by medical professionals. Both physical examination, laboratory testing (
At the end of 2019, an outbreak of COVID-19 occurred. COVID-19 can be transferred from person to person in the world. Data from the World Health Organization (WHO) justify why quarantine action is needed. The WHO declared some classification procedures needed to deal with COVID-19 based on medical images [
With the growing number of infected patients, radiologists find it increasingly difficult to finish the diagnosis process in a limited time [
An overview of some significant works for COVID-19 detection is presented in this section. In [
An alternative modeling system described as DeTraC was suggested in [
In [
High Resolution (HR) image estimation is a complicated assignment developed on the corresponding Low Resolution (LR) image. This process is known as image Super-Resolution (SR). The image SR has gained a significant attention, and has a wide variety of applications. The latest researches have been concentrated on reducing the average square error of restoration. However, these researches often lack concentration on high-frequency information, and their results are perceptually unsatisfactory. Generally, these research works do not satisfy the requirements of SR reconstruction. Hence, in this work, a GAN is introduced for medical image SR to improve the subsequent classification accuracy.
After reviewing the related studies, we can deduce that deep learning can effectively help in the detection of Pneumonia and COVID-19 from CXR and CT images. However, detail enhancement in images has not been considered, deeply. Different medical datasets may display similarity as in the cases of Pneumonia. This, in turn, affects the accuracy of DL classification models. Hence, this paper introduces a hybrid CNN model, namely SIGTra for generating SR versions of COVID-19, Pneumonia, and normal images. It depends on a GAN for the SISR reconstruction problem. Different pre-trained TCNN frameworks (DenseNet121, Densenet169, Dense-Net201 [ A comprehensive study of the classification process of X-ray and CT images is presented. Several sources of images are utilized to distinguish between normal and abnormal cases. Classification process is studied with and without the proposed SIGTra model for SR image reconstruction. A detailed comparison is presented between the different classification models presented in this paper with different training/testing ratios. A comparison is presented between the best classification results obtained with the proposed models on SR images and those of different state-of-the-art models.
The rest of this research work is coordinated as follows. The suggested SIGTra model is presented in Section 2. The simulation and comparison results are presented and analyzed in Section 3. The conclusion and future work are presented in Section 4.
This section presents the proposed hybrid SIGTra model that comprises the SISR based on GAN and the TCNN. It distinguishes between COVID-19 and Pneumonia cases. As demonstrated in
A significant category of image processing techniques in computer vision and image processing is image SR, which belongs to retrieving HR images from LR ones. It has various applications in the real world, such as medical imaging, surveillance, and defense. It also helps to enhance other computer vision functions, apart from enhancing the perceptual image consistency. Since there are often numerous HR images related to a single LR image, this problem is dramatically complicated. A single image SR-GAN algorithm has been suggested based on photo-realistic and natural images [
The deep ResNet [
We define a discriminator network
The general concept behind this formulation is that a generative model (
The concept of perceptual loss
Probable options for the content loss Content Loss
The MSE pixel-wise loss is determined as follows:
where
The MSE is the frequently utilized metric for the optimization of image SR [
Within the DenseNet121 network, Adversarial Loss (GAN Loss)
The generative loss of our SR-GAN model is also added to the perception loss, including the content loss discussed previously. By attempting to deceive the discriminator network, the generator network enhances the features of LR images. The generative loss
where
In our work, several pre-trained CNN and full-training models are employed to examine the robustness and effectiveness of the proposed TCNN model with and without the SR-GAN model. Fine tuning is used to train more layers by changing the parameters of learning until a significant performance improvement is achieved. DenseNet121, Densenet169, Dense-Net201 [
In the simulation scenarios, we change all parameters in the last fully-connected layers in transfer learning through a fine-tuning process. All convolution layers and fully-connected layers remain as in the deep tuning scenario. All these scenarios are discussed to validate the suggested work with and without the GAN-based SISR model. The loss of the model is determined by computing the following sum:
where
The ReLU function is used to replace any obtained negative pixel values with zero. It can be expressed as follows:
where
A large number of medical images is required for successful training and classification, but this is costly. This challenge can be treated through transfer learning, which involves tuning of millions of parameters in CNN architectures. The SIGTra model can be applied on images with similar features, and hence it is adopted in this paper. We apply transfer learning from a generic image recognition task to a medical image classification task.
Our contribution in this paper is the automatic classification of chest CT and X-ray datasets [
For testing the suggested DL frameworks, we used the following image datasets:
COVID-19 CT Dataset Repository on GitHub [ The Data COVID-19 Collection Image Repository on GitHub [ Chest X-Ray Images (Pneumonia) Challenge Detection Dataset [ Extensive COVID-19 X-Ray and CT Chest Images Dataset [
All medical scans are resized to a size of 224 × 224 × 3 or 299 × 299 × 3 according to the employed DL model. Based on the adopted interpolation technique, the appropriate algorithm is selected. The INTER_AREA method is used from the OpenCV library. It allows resampling based on pixel/area relation [
Data augmentation, splitting, and pre-processing procedures are used to expand the training dataset. Feature maps are extracted with the DL models and sent to the multilayer perceptron for classification. The suggested model efficiency is assessed by using the trained network on the test medical images. Every experiment is repeated three times, and then the average outcomes are calculated.
The utilized medical images of all datasets are resized to 224 × 224 except the Xception images, which are resized to 299 × 299 for the classification process. For the proposed SR-GAN model, we begin with images of size 64 × 64. To train the proposed DL models, we test different batch sizes. The validation-to-training ratios are correspondingly set to different values. After several trials to choose the best optimizer, Adam’s optimizer was used due to its high efficiency and short training time. For the classification models, we have used
A drop-out strategy is adopted to decrease the overfitting probability of the employed DL models. The realization of the DL models was accomplished through Kaggle that provides notebook editors with free access to NVIDIA TESLA P100 GPUs and 13 GB RAM operating on Professional Windows Microsoft 10 (64-bit). For simulation tests, Python 3.7 was utilized. In addition, TensorFlow and Keras were employed as DL backend.
Through our assessment, we use accuracy, recall,
The results of the binary and ternary classification are presented in this section for the chest CT and X-ray images with the different DL models including full training of all layers of the proposed CNN model, LeNet-5, AlexNet, VGG16, Inception naïve v1, and Inception v2 with multiple layers. In addition, fine tuning of the top layers of the DenseNet121, Densenet169, DenseNet201, ResNet50, ResNet152, VGG16, VGG19, and Xception is also considered. Additionally, to verify the robustness of all DL models, numerous tests are performed on the chest CT and X-ray scan datasets. We have presented two different scenarios in the simulation results as illustrated in Sub-sections 3.6.1 and 3.6.2. These scenarios are classification with and without implementing the SR-GAN model.
The test or validation curve is obtained based on a validation hold-out dataset. The loss of validation and training is known as the number of miscalculations produced for every instance of training or invalidation. Generally, the best DL model is a model that can be generalized well and that has neither over-fitting nor under-fitting. The confusion matrix reflects the overall performance, as presented in
For simplicity, we present only the curves and confusion matrices for the 80:20 training/testing ratio with the SIGTra model.
From the obtained results on the CT scan dataset in
Model name | Resolution | Train: Test | Accuracy (%) | Loss | Precision (%) | Recall (%) | Log loss | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
X-ray | CT scan | X-ray | CT scan | X-ray | CT scan | X-ray | CT scan | X-ray | CT scan | X-ray | CT scan | |||
CNN | (224,224,3) | 80:20 | 98.49 | 97.65 | 0.0743 | 0.1155 | 99 | 98 | 98 | 98 | 99 | 98 | 0.5210 | 0.8119 |
70:30 | 97.74 | 96.51 | 0.1011 | 0.1183 | 98 | 96 | 98 | 97 | 98 | 96 | 0.7822 | 1.2068 | ||
60:40 | 97.25 | 95.11 | 0.1062 | 0.1460 | 97 | 95 | 97 | 95 | 97 | 95 | 0.9496 | 1.6879 | ||
LeNet-5 | (224,224,1) | 80:20 | 93.47 | 90.77 | 0.1763 | 0.2329 | 93 | 91 | 92 | 91 | 93 | 91 | 2.2542 | 3.1853 |
70:30 | 92.27 | 91.16 | 0.2199 | 0.2447 | 92 | 91 | 91 | 91 | 92 | 92 | 2.6402 | 2.9164 | ||
60:40 | 91.18 | 87.51 | 0.2554 | 0.3512 | 92 | 87 | 90 | 87 | 91 | 87 | 3.0458 | 4.3134 | ||
AlexNet | (224,224,1) | 80:20 | 95.63 | 95.12 | 0.1305 | 0.1586 | 96 | 95 | 96 | 95 | 96 | 95 | 1.5089 | 1.6863 |
70:30 | 93.28 | 92.28 | 0.1785 | 0.3168 | 93 | 92 | 93 | 92 | 93 | 92 | 2.3224 | 2.6664 | ||
60:40 | 93.04 | 90.32 | 0.1972 | 0.2862 | 93 | 90 | 93 | 90 | 93 | 90 | 2.4032 | 3.3445 | ||
VGG16 (Full-Training) | (224,224,1) | 80:20 | 95.47 | 95.66 | 0.1304 | 0.1906 | 96 | 96 | 95 | 96 | 95 | 96 | 1.5649 | 1.4989 |
70:30 | 95.29 | 93.12 | 0.1445 | 0.2369 | 96 | 93 | 96 | 93 | 96 | 93 | 1.6269 | 2.3748 | ||
60:40 | 94.55 | 93.12 | 0.1612 | 0.2840 | 95 | 93 | 94 | 93 | 95 | 93 | 1.8816 | 2.3755 | ||
Inception naïve v1 | (224,224,1) | 80:20 | 94.66 | 84.99 | 0.1478 | 0.3429 | 94 | 85 | 93 | 85 | 94 | 85 | 1.8443 | 5.1839 |
70:30 | 94.25 | 85.16 | 0.1559 | 0.3496 | 94 | 85 | 93 | 85 | 93 | 85 | 1.9871 | 5.1246 | ||
60:40 | 93.12 | 83.08 | 0.1868 | 0.3884 | 93 | 83 | 92 | 83 | 93 | 83 | 2.3752 | 5.8450 | ||
Inception v2 with multiple layers | (224,224,1) | 80:20 | 94.77 | 95.48 | 0.1512 | 0.1727 | 95 | 95 | 95 | 95 | 95 | 95 | 1.8070 | 1.5614 |
70:30 | 94.86 | 94.69 | 0.1416 | 0.1866 | 95 | 95 | 94 | 95 | 94 | 95 | 1.7759 | 1.8332 | ||
60:40 | 94.77 | 92.58 | 0.1472 | 0.2555 | 95 | 93 | 95 | 93 | 95 | 93 | 1.8070 | 2.5630 | ||
DenseNet121 | (224,224,3) | 80:20 | 97.85 | 96.38 | 0.0732 | 0.1143 | 98 | 96 | 98 | 96 | 98 | 96 | 0.7444 | 1.2491 |
70:30 | 96.73 | 93.74 | 0.1068 | 0.1642 | 97 | 94 | 97 | 94 | 97 | 94 | 1.1298 | 2.1639 | ||
60:40 | 96.71 | 89.05 | 0.1093 | 0.2436 | 97 | 89 | 97 | 89 | 97 | 89 | 1.1358 | 3.7820 | ||
DenseNet169 | (224,224,3) | 80:20 | 97.79 | 96.56 | 0.0673 | 0.1079 | 98 | 97 | 98 | 97 | 98 | 97 | 0.7629 | 1.1867 |
70:30 | 96.87 | 95.54 | 0.1041 | 0.1622 | 97 | 96 | 97 | 96 | 97 | 96 | 1.0801 | 1.5397 | ||
60:40 | 96.12 | 88.42 | 0.1238 | 0.3707 | 97 | 88 | 97 | 88 | 97 | 88 | 1.3406 | 4.0008 | ||
DenseNet201 | (224,224,3) | 80:20 | 97.04 | 97.29 | 0.1064 | 0.0901 | 97 | 97 | 97 | 97 | 97 | 97 | 1.0235 | 0.9369 |
70:30 | 96.91 | 95.30 | 0.1128 | 0.1343 | 97 | 95 | 97 | 95 | 97 | 95 | 1.0677 | 1.6229 | ||
60:40 | 96.17 | 91.86 | 0.1174 | 0.1985 | 96 | 92 | 97 | 92 | 97 | 92 | 1.3219 | 2.81311 | ||
ResNet50 | (224,224,3) | 80:20 | 97.74 | 94.58 | 0.0966 | 0.1601 | 98 | 95 | 98 | 95 | 98 | 95 | 0.7950 | 1.8737 |
70:30 | 97.69 | 94.58 | 0.0815 | 0.1361 | 98 | 95 | 98 | 95 | 98 | 95 | 0.7946 | 1.8726 | ||
60:40 | 97.73 | 94.21 | 0.0788 | 0.1404 | 98 | 94 | 98 | 94 | 98 | 94 | 0.7820 | 2.0004 | ||
ResNet152 | (224,224,3) | 80:20 | 98.11 | 96.56 | 0.0801 | 0.1166 | 98 | 97 | 98 | 97 | 98 | 97 | 0.6513 | 1.1866 |
70:30 | 97.38 | 94.22 | 0.0954 | 0.1527 | 98 | 94 | 98 | 94 | 98 | 94 | 0.9063 | 1.9974 | ||
60:40 | 97.23 | 94.48 | 0.1243 | 0.1578 | 98 | 94 | 98 | 95 | 98 | 94 | 0.9496 | 1.9066 | ||
VGG16 (Pre-Training) | (224,224,3) | 80:20 | 95.74 | 94.76 | 0.1275 | 0.2499 | 96 | 95 | 97 | 95 | 96 | 95 | 1.4701 | 1.8113 |
70:30 | 96.95 | 93.62 | 0.0981 | 0.2457 | 97 | 94 | 97 | 94 | 97 | 94 | 1.0553 | 2.2055 | ||
60:40 | 95.96 | 90.59 | 0.1639 | 0.2897 | 96 | 91 | 97 | 90 | 96 | 91 | 1.3965 | 3.2507 | ||
VGG19 | (224,224,3) | 80:20 | 96.39 | 96.75 | 0.1250 | 0.1009 | 97 | 97 | 97 | 97 | 97 | 97 | 1.2468 | 1.1242 |
70:30 | 96.08 | 96.39 | 0.1379 | 0.1029 | 96 | 96 | 97 | 96 | 96 | 96 | 1.3533 | 1.2484 | ||
60:40 | 96.55 | 93.03 | 0.1136 | 0.1958 | 97 | 93 | 97 | 93 | 97 | 93 | 1.1916 | 2.4068 | ||
Xception | (299,299,3) | 80:20 | 94.99 | 85.35 | 0.1952 | 0.3527 | 95 | 85 | 95 | 85 | 95 | 85 | 1.7307 | 5.0590 |
70:30 | 95.26 | 84.46 | 0.1898 | 0.3698 | 96 | 84 | 96 | 85 | 96 | 84 | 1.6388 | 5.3681 | ||
60:40 | 94.15 | 82.35 | 0.1928 | 0.3821 | 94 | 82 | 94 | 82 | 94 | 82 | 2.0202 | 6.0951 |
In this study, the binary classification (COVID-19, Normal) and ternary classification (COVID-19, Normal, and Pneumonia) are investigated on different CT and X-ray scans with transfer learning and full training using recent DL models to compare with the proposed model with SR-GAN. From
Model name | Resolution | Train: Test | Accuracy (%) | Loss | Precision (%) | Recall (%) | Log loss | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
X-ray | CT scan | X-ray | CT scan | X-ray | CT scan | X-ray | CT scan | X-ray | CT scan | X-ray | CT scan | |||
CNN | (224,224,3) | 80:20 | 98.53 | 99.05 | 0.0919 | 0.0496 | 99 | 99 | 99 | 99 | 99 | 99 | 0.5057 | 0.3151 |
70:30 | 98.19 | 97.69 | 0.1071 | 0.0944 | 99 | 98 | 98 | 98 | 98 | 98 | 0.6243 | 0.7964 | ||
60:40 | 97.45 | 95.98 | 0.1323 | 0.1437 | 98 | 96 | 97 | 96 | 98 | 96 | 0.8803 | 1.3865 | ||
LeNet-5 | (224,224,1) | 80:20 | 94.46 | 93.97 | 0.1509 | 0.1771 | 94 | 94 | 94 | 94 | 94 | 94 | 1.9104 | 2.0798 |
70:30 | 93.60 | 92.09 | 0.1771 | 0.2701 | 93 | 92 | 92 | 92 | 93 | 92 | 2.2101 | 2.7311 | ||
60:40 | 92.08 | 89.14 | 0.2012 | 0.2662 | 92 | 89 | 91 | 89 | 91 | 89 | 2.7346 | 3.7501 | ||
AlexNet | (224,224,1) | 80:20 | 96.20 | 96.16 | 0.1190 | 0.1462 | 97 | 96 | 96 | 96 | 96 | 96 | 1.3111 | 1.3235 |
70:30 | 95.22 | 94.89 | 0.1417 | 0.2754 | 95 | 95 | 95 | 95 | 95 | 95 | 1.6482 | 1.7647 | ||
60:40 | 94.11 | 91.87 | 0.1793 | 0.3216 | 94 | 92 | 94 | 92 | 94 | 92 | 2.0322 | 2.8046 | ||
VGG16 (Full-Training) | (224,224,1) | 80:20 | 96.85 | 96.35 | 0.1019 | 0.1389 | 97 | 96 | 97 | 96 | 97 | 96 | 1.0863 | 1.2605 |
70:30 | 96.02 | 95.25 | 0.1290 | 0.2180 | 96 | 95 | 96 | 95 | 96 | 95 | 1.3735 | 1.6387 | ||
60:40 | 95.36 | 93.52 | 0.1445 | 0.2966 | 95 | 93 | 96 | 94 | 95 | 94 | 1.6014 | 2.2374 | ||
Inception naïve v1 | (224,224,1) | 80:20 | 95.77 | 88.32 | 0.1198 | 0.3080 | 95 | 88 | 96 | 88 | 96 | 88 | 1.4609 | 4.0337 |
70:30 | 95.48 | 86.13 | 0.1332 | 0.3459 | 96 | 86 | 96 | 86 | 96 | 86 | 1.5608 | 4.7900 | ||
60:40 | 94.60 | 83.39 | 0.1580 | 0.3916 | 95 | 84 | 95 | 83 | 95 | 83 | 1.8636 | 5.7354 | ||
Inception v2 with multiple layers | (224,224,1) | 80:20 | 95.71 | 96.16 | 0.1231 | 0.1543 | 96 | 96 | 95 | 96 | 95 | 96 | 1.4796 | 1.3235 |
70:30 | 95.19 | 95.86 | 0.1419 | 0.1347 | 95 | 96 | 96 | 96 | 95 | 96 | 1.6607 | 1.4286 | ||
60:40 | 94.98 | 94.16 | 0.1576 | 0.2310 | 95 | 94 | 95 | 94 | 95 | 94 | 1.7325 | 2.0168 | ||
DenseNet121 | (224,224,3) | 80:20 | 98.42 | 96.89 | 0.0773 | 0.0902 | 99 | 97 | 99 | 97 | 99 | 97 | 0.5431 | 1.0714 |
70:30 | 97.07 | 96.84 | 0.1082 | 0.0894 | 97 | 97 | 97 | 97 | 97 | 97 | 1.0114 | 1.0898 | ||
60:40 | 97.20 | 92.79 | 0.0985 | 0.2465 | 97 | 93 | 98 | 93 | 97 | 93 | 0.9646 | 2.4895 | ||
DenseNet169 | (224,224,3) | 80:20 | 97.99 | 96.71 | 0.0807 | 0.1005 | 98 | 97 | 98 | 97 | 98 | 97 | 0.6930 | 1.1344 |
70:30 | 97.54 | 96.11 | 0.0933 | 0.1051 | 97 | 96 | 98 | 96 | 98 | 96 | 0.8491 | 1.3413 | ||
60:40 | 97.23 | 93.79 | 0.0918 | 0.1985 | 97 | 94 | 98 | 94 | 97 | 94 | 0.9552 | 2.1429 | ||
DenseNet201 | (224,224,3) | 80:20 | 97.45 | 97.81 | 0.0881 | 0.0760 | 98 | 98 | 98 | 98 | 98 | 98 | 0.8803 | 0.7563 |
70:30 | 97.07 | 97.20 | 0.1021 | 0.0899 | 97 | 97 | 97 | 97 | 97 | 97 | 1.0114 | 0.9640 | ||
60:40 | 96.58 | 95.34 | 0.1116 | 0.1646 | 97 | 95 | 97 | 95 | 97 | 95 | 1.1800 | 1.6071 | ||
ResNet50 | (224,224,3) | 80:20 | 98.10 | 96.71 | 0.0828 | 0.0891 | 98 | 97 | 98 | 97 | 98 | 97 | 0.6555 | 1.1344 |
70:30 | 98.08 | 96.60 | 0.0749 | 0.1061 | 98 | 97 | 99 | 97 | 98 | 97 | 0.6618 | 1.1736 | ||
60:40 | 97.26 | 95.62 | 0.0911 | 0.1225 | 98 | 96 | 98 | 96 | 98 | 96 | 0.9458 | 1.5126 | ||
ResNet152 | (224,224,3) | 80:20 | 98.21 | 97.99 | 0.0731 | 0.0760 | 98 | 98 | 99 | 98 | 98 | 98 | 0.6181 | 0.6932 |
70:30 | 97.93 | 96.60 | 0.0864 | 0.1045 | 98 | 97 | 98 | 97 | 98 | 97 | 0.7117 | 1.1736 | ||
60:40 | 97.72 | 95.71 | 0.1039 | 0.1681 | 98 | 96 | 98 | 96 | 98 | 96 | 0.7866 | 1.4811 | ||
VGG16 (Pre-Training) | (224,224,3) | 80:20 | 97.12 | 96.16 | 0.0983 | 0.1470 | 97 | 96 | 98 | 96 | 97 | 96 | 0.9927 | 1.3235 |
70:30 | 96.85 | 96.11 | 0.1075 | 0.1224 | 97 | 96 | 97 | 96 | 97 | 96 | 1.0863 | 1.3413 | ||
60:40 | 96.23 | 95.62 | 0.1395 | 0.1280 | 96 | 96 | 97 | 96 | 97 | 96 | 1.3017 | 1.5126 | ||
VGG19 | (224,224,3) | 80:20 | 97.72 | 97.81 | 0.0818 | 0.0892 | 98 | 98 | 98 | 98 | 98 | 98 | 0.7866 | 0.7563 |
70:30 | 97.36 | 97.33 | 0.1090 | 0.0825 | 98 | 97 | 98 | 97 | 98 | 97 | 0.9115 | 0.9221 | ||
60:40 | 96.69 | 95.71 | 0.1321 | 0.1468 | 97 | 96 | 97 | 96 | 97 | 96 | 1.1425 | 1.4811 | ||
Xception | (299,299,3) | 80:20 | 95.77 | 93.06 | 0.1457 | 0.1985 | 96 | 93 | 95 | 93 | 96 | 93 | 1.4609 | 2.3950 |
70:30 | 95.51 | 92.71 | 0.1520 | 0.1848 | 96 | 93 | 96 | 93 | 96 | 93 | 1.5483 | 2.5149 | ||
60:40 | 94.22 | 89.05 | 0.2697 | 0.3003 | 94 | 89 | 94 | 89 | 94 | 89 | 1.9947 | 3.7816 |
To further prove the classification efficacy of the suggested SIGTra model, we compared it with other recent related works as presented in
Model name | Method | Modality | Accuracy (%) | Precision (%) | Recall (%) | |
---|---|---|---|---|---|---|
EfficientNet-B3-GAP [ |
COVID19-CT | CT | 88.18 | 88.18 | 88.18 | 88.15 |
A comprehensive study [ |
ResNet-50 | X-ray | 97.50 | 95.24 | 100.00 | 98.36 |
ResNet-50 | 96.67 | 96.67 | 96.67 | 96.67 | ||
VGG-16 | 97.50 | 96.72 | 98.33 | 97.52 | ||
Semisupervised Adversarial [ |
DenseNet121 | CT scan | 92.0 | – | – | – |
VGG16 | 93.33 | – | – | – | ||
ResNet50 | 99.0 | – | – | – | ||
COVID-19-Net | 98.45 | – | – | – | ||
CovidGAN [ |
CNN-AD | X-ray | 85 | 95 | 69 | – |
CNN-SA | 95 | 97 | 90 | – | ||
DeTraC [ |
AlexNet | X-ray | 95.66 | 93.49 | 97.53 | – |
VGG19 | 97.35 | 96.34 | 98.23 | – | ||
ResNet | 95.12 | 91.87 | 97.91 | – | ||
Multi-task pipeline [ |
Inception-v3 | CT scan | 98.1 | – | – | – |
Shrunken features [ |
Feature vectors | X-ray | 86.54 | 86.35 | 83.15 | 84.55 |
SAE | 71.92 | 69.89 | 68.91 | 69.13 | ||
PCA | 94.23 | 96.73 | 91.88 | 93.99 | ||
Gamma Mixture [ |
gIMM-FD | X-ray | 94.08 | – | – | – |
IMM-FD | CT scan | 84.23 | – | – | – | |
Multiscale Attention Guided [ |
VGG16 | X-ray | 90.68 | 94.90 | 91.05 | 89.44 |
ResNet | 93.41 | 96.26 | 93.71 | 93.12 | ||
MAG-SD | 95.85 | 97.73 | 95.74 | 95.54 | ||
Semi-supervised Shallow Learning [ |
ResNet50 | CT scan | 98.4 | 98.3 | 98.6 | 98.5 |
Semi-supervised Shallow | 98.4 | 98.6 | 98.5 | 98.3 | ||
Deep Learning Approaches [ |
ResNet50 Features + SVM | X-ray | 95.79 | 97.78 | 94.00 | 95.92 |
Fine-tuning of ResNet50 | 92.63 | 97.78 | 88.00 | 92.63 | ||
BSIF + SVM | 91.58 | 93.33 | 90.00 | 91.84 | ||
Discriminant correlation analysis [ |
CCSHNet | CT scan | – | 97.03 | 97 | 97.02 |
SIGTra (Proposed work) 2021 | SR-GAN+TCNN | CT scan | 99.05 | 99 | 99 | 99 |
X-ray | 98.53 | 99 | 99 | 99 |
This paper presented a proposed hybrid SIGTra model to classify chest X-ray images and CT scans into Normal, COVID-19, and Pneumonia classes. The proposed SIGTra model is used to improve the classification process with an SISR stage based on the SR-GAN. The paper introduced comprehensive comparisons between different DL models including the proposed model, LeNet-5, AlexNet, VGG16, Inception naïve v1, Inception v2 with multiple layers, DenseNet121, DenseNet169, DenseNet201, ResNet50, ResNet152, VGG16, VGG19, and Xception. Several experimentations have been performed on chest X-ray and CT images. The proposed SIGTra model leads to superior results compared to other DL models. Future research may include developing a complete Pneumonia classification system through deep learning, super-resolution, and classification. Moreover, the classification process can be performed on more datasets, with more advanced techniques. Efficient sub-pixel and Very Deep SR (VDSR) can also be considered as new tools for SISR.
The authors would like to thank the support of the Deanship of Scientific Research at Princess Nourah Bint Abdulrahman University.