X-Ray Covid-19 Detection Based on Scatter Wavelet Transform and Dense Deep Neural Network

Notwithstanding the discovery of vaccines for Covid-19, the virus's rapid spread continues due to the limited availability of vaccines, especially in poor and emerging countries. Therefore, the key issues in the present COVID-19 pandemic are the early identification of COVID-19, the cautious separation of infected cases at the lowest cost and curing the disease in the early stages. For that reason, the methodology adopted for this study is imaging tools, particularly computed tomography, which have been critical in diagnosing and treating the disease. A new method for detecting Covid-19 in X-rays and CT images has been presented based on the Scatter Wavelet Transform and Dense Deep Neural Network. The Scatter Wavelet Transform has been employed as a feature extractor, while the Dense Deep Neural Network is utilized as a binary classifier. An extensive experiment was carried out to evaluate the accuracy of the proposed method over three datasets: IEEE 80200, Kaggle, and Covid-19 X-ray image data Sets. The dataset used in the experimental part consists of 14142. The numbers of training and testing images are 8290 and 2810, respectively. The analysis of the result refers that the proposed methods achieved high accuracy of 98%. The proposed model results show an excellent outcome compared to other methods in the same domain, such as (DeTraC) CNN, which achieved only 93.1%, CNN, which achieved 94%, and stacked Multi-Resolution CovXNet, which achieved 97.4%. The accuracy of CapsNet reached 97.24%.


Introduction
Ended in December 2019, a cluster of pneumonia cases with an unknown cause was linked to a seafood market in Wuhan, Hubei Province, China, and quickly becoming a pandemic [1]. The virus was isolated by Scientists from the patient's airway epithelial cells. Furthermore, other than SARS-CoV, influenza, avian influenza, MERS-CoV, and numerous respiratory viruses, it has been formally designated as severe acute respiratory syndrome coronavirus 2 (SARSCoV2) [2,3], SARS-CoV-2 [4]. Initially, this new virus was called 2019-nCoV [5][6][7]. On Feb 11, 2020, the virus was renamed Covid-19 by the World Health Organization (WHO) [8]. Since the first identification in the 1960s, Coronavirus has been responsible for Artificial Intelligence (AI) and related technologies are increasingly prevalent in healthcare and medical practice. It is relevant to visually-oriented specialty, such as radiography. This field has got particular attention; the reason is the wide use of x-ray examination, which is around two billion performed per year. AI techniques are also deeply employed in Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) to help the medical staff diagnose and treat diseases [33][34][35]. Today, the world is facing a major health crisis because of the pandemic of Covid-19; the medical industry is working hard to find and utilize new technologies to help medical staff diagnose and control diseases. AI is one such technology that enables rapid monitoring of the virus's spread, classification of high-risk patients and realtime infection control. Additionally, it can forecast mortality risk by fully evaluating a patient's prior outcomes. AI will support us in combating this illness by monitoring the community, offering medical support, issuing warnings and making infection prevention advice [36]. The Covid-19 is a serious contagious disease and rapidly spreading, and it has had disastrous consequences on both daily life and the global economy. One of the best ways to prevent and reduce it is by identifying positive cases as early as possible. Furthermore, the sooner we find those cases, the sooner we can treat them.
Real-time reverse-transcription polymerase chain reaction (RT-PCR), X-ray and CT are utilized to diagnose the Covid-19 to distinguish between infected and non-infected cases. The reports have found that RT-PCR has a lower sensitivity compering to CT of detect Covid-19 [37][38][39]. The DarkNet model was used as a classifier to detect and classify the chaste x-ray images for suspected COVID-19 cases. The accuracy of the system was 98.08% for binary classes [40]. An automatic framework based on deep learning has been developed to detect Covid-19 using chest CT [41]. Chest CT scan is an important and most common method that has been used to identify pneumonia. Gozes, O. et al. have been developed AI-based automated CT image analyzer tools to detecting, quantifying and tracking the Covid-19 in order to differentiate between positive and negative cases. The possible working point was 98.2% sensitivity and 92.2% specificity [42]. Both Canonical-Correlation Analysis (CCA) and Support Vector Machine (SVM) were implemented to detect Covid-19, CCA was used as a feature selection, while SVM was used as a classifier. The performance was excellent, with 98.27% accuracy, and the sensitivity was 98.93% [43].
Ahmmed K. et al. (2020) begin with preprocessing the images of x-ray by converting them into grayscales and resize them into 100 Â 100. The proposed Convolutional Neural Network (CNN) begins with 32 filters, then 64 filters, and max pooling filters. The result input is in a dense neural network for the classification of the image. The chest X-ray images were obtained from the Covid-19 Radiography Database, which was considered as a primary dataset in this work. This dataset contained 1341 normal and 219 Covid-19 patient's images. The accuracy obtained from that model was 94% [44].
The study of Asmaa A. et al. (2020) validates Decompose, Transfer and Compose (DeTraC), a deep CNN for COVID-19 chest X-ray image classification. DeTraC can cope with anomalies in the picture dataset by employing a class decomposition process to investigate its class boundaries. The experimental results demonstrated DeTraC's capacity to detect COVID-19 cases from a large image dataset, and the accuracy gained for this approach was 93% [45]. Different types of CovXNets are built and trained by Mahmud T. et al. (2020) with X-ray images of varying resolutions in the proposed method, and a stacking strategy is used to optimize their predictions. Finally, a gradient-based discriminative localization is implemented to identify the aberrant regions of X-ray pictures relating to distinct forms of pneumonia. One of the datasets included in this investigation is a collection of 5856 images obtained in Guangzhou Medical Center, China, consisting of 1583 normal X-rays, 1493 non-COVID viral pneumonia X-rays and 2780 bacterial pneumonia X-rays. Another dataset was provided by Sylhet Women's Medical College/ Bangladesh. It contains 305 X-rays of distinct Covid-19, was acquired and confirmed by an expert radiologist. Accuracy gained from the proposed model is 97.4% [46]. The study of Toraman S. et al.
(2020) uses chest X-ray images and capsule networks, and this study proposes a unique artificial neural network and Convolutional CapsNet, to identify COVID-19 disease. CpsNet is made up of five layers of convolutions. The first layer has 16 kernels of size 5 Â 5 with a stride of 1. The first layer exit was max pooled with a size of 2 and a stride of 2. The next structure was used in the next two layers. The second and third levels' kernel numbers are 32 and 64, respectively. The fourth layer contains 128 kernels with a stride of 1. The fifth layer is the major one, which comprises 32 capsules, each of which has applied 9 convolutional kernels with a stride of 1. The LabelCaps layer has 16-dimensional (16D) capsules for two and three classes, and all layers use the Rectified Linear Activation Function (ReLU). The dataset used in this study was collected by Cohen, and the accuracy achieved was 97.24% [47]. Several chest x-ray pictures from diverse sources are gathered by Haghanifar A. et al. (2020), and the largest publicly accessible collection was created. Finally, COVID-CXNet was developed using the transfer learning paradigm and the well-known CheXNet model. Based on relevant and meaningful variables and exact localization, this powerful model can recognize the new coronavirus pneumonia. COVID-CXNet is a step toward a COVID-19 detection system that is entirely automated and reliable. The accuracy for base model version I was 96%, and for base model version II was 98% [48]. The study of  analyses the backbone network. In this article, the backbone network was an 18-layer residual CNN that was pre-trained on the ImageNet dataset. The rectangles with different colors in Fig. 1(a) depict the five stages of the backbone network. In the first stage, large convolutions with 7 Â 7 kernels and a stride of 2 are utilized, followed by a 3 Â 3 max-pooling layer with a stride of 2. Following that, each stage comprises two residual blocks, each with two convolutional layers and one skip connection. The input image can be encoded as a feature map with an output stride after layer-by-layer convolutional procedures. The accuracy gained was about 0.96 to detect Covid cases and 70% for detecting non-Covid cases [49]. The proposed work of Das A.K et al. (2020) uses many CNN models, including DenseNet201, Resnet50V2 and Inceptionv3. Individually, they have been trained to make predictions on their own. The models are then merged to forecast a class value using a novel weighted average ensembling technique. The study used publicly available chest X-ray pictures of COVID and normal instances to test the solution's efficacy. Training, test and validation sets were created using 538 photos of COVID patients and 468 images. The proposed method achieved an accuracy of 91.62 percent, higher than both current CNN models and the benchmark algorithm [50]. Narin A et al. (2021) have applied five different pre-trained CNN-based models, which are ResNet101, ResNet50, ResNet152, Inception-ResNetV2 and InceptionV3. For the detection of coronavirus pneumonia-infected patients, the chest X-ray radiographs were proposed in this study. The average accuracy from tests was 0.98 [51]. This paper answers two questions; first, is a scattering wavelet an effecting feature to detect Covid-19 from x-ray images? While the second question is that scattering wavelet represents a discriminant feature that can build a classifier trained with a small dataset? This research aims to build a model that can detect the infected cases of Covid-19 from X-ray images.
The rest of the paper has been classified as follows: Section 2 discusses the wavelet scattering transform, while the dense deep neural network is debated in Section 3. The proposed technique is given in Section 4. Finally, the conclusion is presented in Section 5.

Scattering Wavelet Transform (SWT)
Since the capability of wavelets for multi-resolution image representation, the interest in using wavelet frames for image processing has grown over recent decades. The framelet transform is close to wavelet transform Framelet has two high-frequency filters, which produce more sub bands in decomposition. Convolutional structure of the windowed scattering transform means that each layer is captured from previous by applying wavelet modulus decomposition U on each envelope p ½ f . The output layer is obtained by hj. Scattering is chosen in this research because it calculates iteratively by applying U, inverting Sj requires to invert U so it can catch the texture coefficients iteratively. Despite the importance of spectral methods, the Fourier spectrum is often not sufficient to discriminate image texture because it does not account for high order mount. The efficiency of scattering representation in discriminate texture is clear in Fig. 1, which has the same power spectrum with the same second-order moments. Scattering coefficients, Sj p ½ x are shown in Fig. 1 with frequency tiling. So, for texture feature extraction, scattering wavelet is more efficient than wavelet transform, discrete wavelet transform and framelet wavelet transform [52].
The SWT was early presented by Mallat [53] in 2012. The SWT constructs translation invariant, stable and informative signal representations [54]. It is stable in deformations and maintains class discrimination, especially successful in classification [55]. The SWT provides a handy classification tool [56][57][58][59]. Reliable information can be extracted at various scales by SWT, the coefficients of SWT are more descriptive than the Fourier while interacting with small deformation and rotation invariant signals [60].
The SWT network consists of two layers without training and achieves similar performance to the convolutional network. The network generates a representation Φ that is invariant to rotation, translation, color discrimination and stability to small deformations. A scattering transform is the cascading of linear wavelet transform and modulus nonlinearity.
The image is filtered using the first wavelet transform W 1 , Eq. (1). A complex wavelet that has been rotated and scaled, as shown in Fig. 2, then producing the modulus of each coefficient to define the first scattering layer U 1 y. Consequently, the next layer comes to compute U 2 y by using the modulus of W2. Scattering coefficient S 3 x results from a final pooling. The pooling is average-pooling (AVG) or maxpooling defined by blocks of size.
The initial wavelet layer is described by the base wavelet ψ1 (u), a complex function with excellent image plane localization. This wavelet is scaled by 2j1 where j is an integer and rotated by h1 ¼ 2kp=K for 0 k , K that : This wavelet transform is a filter on image x(u), and wavelet coefficients are computed by The spatial variable u would be subsampled by 2j1 À 1: The aggregated variable, which indexes the first layer coefficient, is written by u1 ¼ u; w1 ð Þ. The next layer is computed using a second wavelet transform that is convoluted the u1 coefficient by u1x(u1) with a separated wavelet and spatial rotation, scale, variables The index w2 ¼ j2; k2; l2 ð Þrepresents the values of 2j2, 2k2 and 2l2 of these wavelets. This wavelet type is chosen to define a robust frame, and invertible linear operator saves the norm. This wavelet type contains the average filters that are necessary. The next layer coefficient is described as u1 ¼ u; w1 ð Þ and 3 Deep Dense Neural Network (DDNN) Deep Learning (DL) has its origins in Artificial Intelligence and has emerged as a distinct and highly useful area in recent years [61]. DL architectures in Fig. 3 include models consisting of an input layer, the output layer and two or even more hidden layers of nonlinear processing units, which conduct the extraction and transformation functions that are supervised or unsupervised [62]. These layers are placed in a hierarchical pattern, with each layer's input being the preceding layer's output, computed using a suitably chosen activation function. Each hidden layer represents a distinct image of the input space and provides a different abstraction level that is ideally more useful and closer to the eventual objective [63]. One of the most effective neural networks is Dense Deep Neural Network. It consists of several fully connected layers. It is defined as a function that ranges from R m to R n . Each dimension of the output depends on each dimension of the input [64]. The fully connected layer is depicted pictorially. The converting procedure in image processing is commonly employed, where localized features are combined with the kernel to generate new outputs. The parameters in convolutional layers are determined during the training process, unlike in classic image processing approaches, where the kernel parameters are predefined [65].
The mathematical form of a fully connected network is x 2 R m that represents the input to a fully connected layer. Let yi 2 R be the i À th output from the fully connected layer. Then yi 2 R is computed as follows: Here, σ is nonlinear, and the wi is learnable parameter in the network. The full output y is then while the weights w(l)ij of connection lines between node j in the layer number I and another node in layer l þ 1 layer, the inputs of a dataset are x1,x2,…,x14. a l ð ÞI is the output of the i node in the l layer. Thus, the construction of the DDNN is obtained from the following equations: where f 1; f 2 correspond to the sigmoid and softmax activation function respectively and are defined as follows: Relu usually has been used in deep dense neural networks. Relu function can be defined as follows: Both the Relu function and its derivative are consistent. If the function receives any negative input, it returns 0; however, it returns that value if the input is positive. As a result, it produces an output from a range of 0 to infinity. Fig. 4 depicts the behavior of the Relu function.

Proposed Method
An active and accurate Covid-19 detection has been proposed based on the Scatter Wavelet Transform and DDNN. The SWT has been employed as a feature extractor, while the DDNN is utilized as a binary classifier. Firstly, it extracts all scattering wavelet feature sets from an image pattern. The scattering The converted images or the list of scattering wavelet images are divided into training and testing set of 80% and 20%, respectively. The values produced from the scattering wavelet transformation process are over 1, so these values have to be normalized and converted in decimal format between 0 and 1 to be accepted by DDNN. The DDNN classifier consists of five layers. First, the input layer that is flattening the input list of 7812 features of each x-ray image; then three hidden layers with Relu functions consist of 256,128,64 neurons, respectively; lastly, the output layer that has one neuron with sigmoid activation function, which yields wither normal lung or infected one. DDNN architecture of the proposed system is apparent in Fig. 6.
This Deep Dense Neural Classifier uses Adam that is short of adaptive moment estimation, a momentum-based optimizer. The loss function used is binary cross entropy. The binary classification problems give output in the form of probability. Binary cross entropy is usually the optimizer of choice. Fig. 7 presents the block diagram of the proposed method.

Experimental Results
To test the proposed framework, this research has used three datasets for training, testing and evaluating the proposed method that has been developed to detect Covid-19 infections intelligently. The existing datasets have the disequilibrium issue. This issue is handled by class weighting within the training process. Tab. 1 represents the dataset with the number of images used for testing in both Covid-19 and normal classes.  The datasets vary between big datasets as Kaggle [66], smaller IEEE 80200 [67], to tiny ones the COVID-19 X-ray Image Data Sets. The reason behind choosing these three datasets was to test the proposed system in datasets with different sizes to check the efficiency of that system, which depends on scattering wavelet features as a discriminative feature in small and big datasets. Using the small datasets confirms the claims that scattering wavelets can be a good discriminative between classes for datasets with a small training/testing set.
Five metrics that measure system efficiency in this paper are: accuracy, precision, recall, sensitivity and specificity. These five metrics are derived from four parameters: true positive (TP), true negative (TN), false positive (FP) and false-negative (FN). The formulas of these metrics are in equations from 16 to 19: The features were extracted using scattering wavelet transform, while the proposed system used DDNN as a binary classifier to classify the input x-ray images to either be detected with Covid-19 or not. The DDNN is a fully connected deep network that use a linear operation where every input is connected to every output by a wight. The system is trained using adaptive moment learning rate (ADAM) with epochs between 60 and 100 that is using the early stop in keras library that makes the training process stops when the model gets the best accuracy.
The results show that using scattering wavelet feature with deep dense neural networks classifier achieved high performance. The performance of the proposed model is clear for both small datasets like the third dataset of Covid-19 and larger ones like Kaggle and IEEE 8023. From Tab. 2, the results can be understood. For IEEE 8023 dataset, the accuracy, in general, is 0.98. The sensitivity is 0.96 in general. The sensitivity for detecting positive cases is excellent, and recall has been reached 1.00 in detecting Covid cases. The specificity registering a score of 1.00 showing an excellent in avoiding false detection or false positive. False-positive reached zero, which means that the system did not make any wrong  Fig. 8b shows the number of false-negative only one from 45 x-ray pictures in the testing set. Also, false-positive is zero in Fig. 8b, which represents an indicator of the high performance of the proposed method. In the Kaggle dataset, the accuracy was 0.97 while the sensitivity was 0.94 that is a good accuracy for an imbalanced dataset with a big difference between number of normal chest images and number of infected images. This percentage is clear in the confusion matrix in Fig. 9b with 685 real detected Covid-19 cases. Specificity was also very good, with about 0.98 equal to the precision of detecting normal cases, which are the negative cases (normal cases), which is also registered in the confusion matrix in Fig. 10b with 1985 of true negative cases. Lastly, it analyzes the smallest Covid-19 online dataset. The results show a general accuracy of 97%. The sensitivity of 93% shows that the system is working efficiently to detect Covid cases, Fig. 10b confusion matrix shows that true positive is 13 while the false positive is just 1 case. The confusion matrix registers 13 cases for true positive and 19 cases for true negative, making 19 cases of 30 total testing set images. These values indicate a high performance for the proposed model trained with a small dataset of x-ray images. The sensitivity is clearly accommodating the precision of Covid detection. The specificity of 94% represents the precision of catching the normal cases. The amount of specificity of 94% may be small due to the small dataset consisting only of 64 images for training.  In the discussion section, the proposed model has been compared with previous related works; this is shown in Tab. 3. The proposed model relying on feature extracted by scattering wavelet and deep dense neural network has shown a good accuracy of over 96.5% on average. However, when using the models that depend on only convolution neural networks individually, the accuracy was lower than the reported values, except Ozturk et al., who used Darknet CNN. Abbas et al., for example, used a small dataset of 105 Covid-19 images and 80 normal images, trained a transferred DecTrac CNN model and got an accuracy about 93.1%, which is relatively low accuracy. The CNN model defiantly needs larger datasets to be trained with acceptable accuracy, and this also tested within the papers that used pre-trained model  and mixed models that used more than one pre-trained model in papers [50] that used a Merge of (Densenet201, Resnet50v2, inceptionv3) and achieved only 91% of accuracy and [51] who proposed a model of five pre-trained model that reached to the same accuracy of proposed model.
The proposed model tested with the mentioned datasets. The resulted accuracy values are answering the two mentioned research questions in the introduction section. First, is scattering wavelet is an efficient? The accuracy values were achieved in the tests indicate that scattering wavelet transform is an efficient enough to train covid-19 detection model.
The second research question was whether the scattering wavelet efficient as a discriminant feature to build the covid-19 detection system with a small training set?
The system also tested with mentioned second dataset and the accuracy was high and stable with 97%.

Conclusion
Because of the vast number of deaths caused by the coronavirus pandemic, healthcare services in every country have been stretched to their limits. COVID-19 can be detected early and treated more quickly, simpler and less expensively will save lives and relieve the pressure on medical providers. The image processing techniques and artificial intelligence to X-ray images will play a major role in identifying COVID-19. This research has crafted an intelligent framework for identifying COVID-19 with high accuracy and minimal complexity. The present research aims to investigate the use of scattering wavelet as a discriminative feature to detect Covid-19 from x-ray images supported by deep learning technique. The proposed system was applied to three different datasets that are varying in size. The first dataset consists of 231 pictures, the second dataset contains 13808 x-ray pictures, and the third dataset consists of 112 pictures. The proposed system shows a stable efficiency even with a small dataset. The results insisted that the scattering wavelet is efficient in providing discriminative features about the internal patterns of images even in the case of small dataset availability. Moreover, the proposed work employed the effectiveness of deep learning of classification.
These training and testing images represent all of the images from the previous experiments. The results of the present study are compared to other methods in the same domain. It is identified that the proposed method has achieved an accuracy of 98% when compared to (DeTraC) CNN, which achieved only 93.1%, CNN, which achieved 94% and stacked MultiResolution CovXNet, which achieved 97.4%. The accuracy of CapsNet reached 97.24%. For future studies, the area of detecting Covid-19 from x-ray images needs much more patient datasets, well organized, balanced and standard, because current datasets have issues like unbalancing between classes or some mistakes like normal cases images stored accidentally in Covid-19 patient classes and vice versa.
Funding Statement: The authors received no specific funding for this study.

Conflicts of Interest:
The authors declare that they have no conflicts of interest to report regarding the present study.