Symbiotic Organisms Search with Deep Learning Driven Biomedical Osteosarcoma Detection and Classification

1  Introduction

Osteosarcoma or Osteogenic Sarcoma is a type of bone cancer that commonly starts developing from long bones of arms and legs [1]. Though its growth can be observed in any of the bone cells within bones, it rarely affects the soft tissues adjacent to bones. Though it has been prevalently diagnosed among teenage or young people, it also affects other age categories. Multiple reasons are cited behind the occurrence of osteosarcoma. Alike other cancer types, it occurs due to genetic disorder and is not mostly connected to familial traits [2]. However, in one of the studies conducted earlier, osteosarcoma was found to have association with familial retinoblastoma cases i.e., family history is concerned with adolescent cancer in eye [3]. The symptoms of osteosarcoma disease include aches from the affected bone in the beginning while as time goes by, the low level of pain occurs and the patient’s agony increases. Bone fracture is another important symptom since the affected bones become weak owing to infection and lose its strength [4]. Magnetic Resonance Imaging (MRI), X-rays and histological biopsy test are necessary for diagnosis of osteosarcoma. In recent times, the prognosis of osteosarcoma demands a comprehensive history of the patient and their physical checks [5]. The common indications include constant gnawing pain and swelling at the site of infection, deep-seated, etc., ache in many parts which might portend skeletal metastasis; thus, these symptoms must be properly examined [6].

Earlier, the histopathological images are scrutinized physically or through image processing technologies to find the clusters of nuclei, osteoblasts, development plates, fibrous tissue, blood vessels, nuclei sizes, calcified bone sections, adipocytes, cartilage, marrow cells and osteoid. This is not only a time-consuming process, but also tend to err [7]. At present, advanced Artificial Intelligence (AI) techniques can be applied for cancer cell recognition using simple datasets. The features can be identified automatically within a limited period of time. Additionally, the utilization of AI offers good support for patients as well as doctors by prognosing the symptoms [8]. Besides, the utilization of AI in histopathology domain helps in achieving superior prognosis, categorization, and identification of distinct forms of cancer [9]. At present, Computer-Aided Detection (CAD) algorithms have been developed by researchers to diagnose osteosarcoma through medical images. These techniques particularly make use of Computed Tomography (CT) and MRI scan images. But the outcomes from such systems are confined owing to constraints found in CT and MRI scans [10].

The authors in the literature [11] executed a pipeline for automatic segmentation of Region of Interest (ROI). In this study, a nomogram was utilized combining MRI-based radiomics’ score and medicinal variables so as to predict the responses for Neo-Adjuvant Chemotherapy (NAC) among osteosarcoma patients. After normalization, ROIs is segmented by DL segmentation method to preoperative MRI and is trained with nnU-Net using two independent manual segmentation labels. Radiomics feature is employed to automatically extract the segmented ROIs. In literature [12], the authors presented an automatic recognition model named Integrated Features-Feature Selection Model for Classification (IF-FSM-C) which diagnoses osteosarcoma in high-resolution Whole Slide Images (WSIs). In order to perform FS, two binary variations were used for the newly-presented Arithmetic Optimization Algorithm (AOA) such as BAOA-S and BAOA-V. The chosen features were then provided for classification during when the WSI is classified as Non-Tumor (NT), Viable Tumor (VT), and Non-Viable Tumor (NVT).

In literature [13], a new and more important structure W-net++ was presented based on two cascaded U-Nets and dense skip links so as to realize automatic and correct segmentation of osteosarcoma lesion from CT scan image. In this network, multiscale input was used to structure the missing spatial details, due to several encoders and encoding subsamples. In the study conducted earlier [14], Transfer Learning (TL) approaches like pre-trained Convolutional Neural Network (CNN) were altered in public data set on osteosarcoma histological image to detect the necrotic image in non-necrotic and healthy tissues. At that time, TL techniques that contain Visual Geometry Group (VGG19) and Inception V3 were utilized and trained on WSIs without patches so as to enhance the accuracy of output. The authors in literature [15] presented CNN as a tool to enhance the efficacy and precision of osteosarcoma tumor classification under tumor classes (VT, necrosis) and non-tumor classes.

The current study introduces a novel Symbiotic Organisms Search with Deep Learning-driven Osteosarcoma Detection and Classification (SOSDL-ODC) model. The presented SOSDL-ODC technique applies image pre-processing approach to enhance the quality of image. In addition, MobileNetv2 model is applied to generate a suitable group of feature vectors and the hyperparameter tuning of MobileNetv2 model is performed using SOS algorithm. At last, Gated Recurrent Unit (GRU) technique is applied as a classification model to determine suitable class labels. In order to validate the enhanced osteosarcoma classification performance of the proposed SOSDL-ODC technique, the authors conducted a comprehensive comparative analysis and the results were validated.

2  Materials and Methods

In this work, a new SOSDL-ODC approach has been developed for recognition and classification of osteosarcoma using histopathological images. The presented SOSDL-ODC system initially applies image pre-processing approach to enhance the quality of image. After image pre-processing, SOS-MobileNetv2 model is applied to generate a suitable group of feature vectors. Finally, GRU method is applied as a classification model to determine the appropriate class labels. Fig. 1 depicts the overall block diagram of SOSDL-ODC technique.

Figure 1: Block diagram of SOSDL-ODC technique

2.1 Image Pre-processing

The presented SOSDL-ODC technique initially applies image pre-processing approach to enhance the quality of image. Wiener Filter (WF) is utilized in this study to filter noisy signals. This is achieved by employing the spectral feature of the chosen signals whereas the noise present in the signals is assumed to undergo stochastic procedures with linear properties.

Linear filter is executed with a co-efficient WK on the evaluated signals. An input signal, x(n) contains noises as follows v(n),

x(n)=d(n)+v(n)(1)

The resultant signal y(n) is a close evaluation of (n). Therefore, the error signal e(n) is lesser. The adaptive method attempts at correcting the weighted WK, and so the Mean Square Error (MSE) value is minimized.

e=min(E(e(n)2))(2)

whereas,

e(n)=y(n)−d(n)(3)

A k tap discrete WF utilizes the subsequent formula to determine the value of y (n)

y(n)=∑k=0N−1Wk(d(n−k)∗v(n−k))(4)

Wiener-Hopf formula that computes the optimum weight is one of the important features of WFs.

El=0p−ιWolrxx(k−1)=rxd(−l)(5)

Here, W00,W01,⋯⋯⋅⋅,Wop−1 denotes the optimal value of tap weighted filters and rxx signifies the autocorrelation function of x(n) and rxd denotes the cross correlation function between x(n) and d(n).

2.2 Feature Extraction: MobileNetv2 Model

After image pre-processing, MobileNetv2 technique is leveraged to generate a suitable set of feature vectors [16]. MobileNetV1 is developed based on conventional VGG structure and is used to construct a system by stacking the convolutional layers so as to enhance the performance. But, gradient vanishing problem arises after stacking multiple convolution layers. The residual blocks present in ResNet make it easy for the dataset to stream among the layers. This process gets rid of the gradient exploring problems in backpropagation while the features are reused in forwarding propagation. In addition to continuing with in-depth separable convolutional layers of MobileNetV1, the MobileNetV2 technique draws on ResNet architecture too. In comparison with MobileNetV1, the major development found in MobileNetV2 is that it comprises of two points such as the accomplishment of inverted residual block and linear bottleneck in the networks. A certain conceptual framework feature is attained from the expansion factor t in N to M networks with stride, s. This bottleneck includes a 1 × 1 convolutional layer before depth-wise convolution layers and employs linear activation instead of non-linear activation after point-wise convolution layers. Further, it determines the down-sampling process by setting the variable, ‘s’ in depth-wise convolution layers.

In the entire architecture of MobileNetV2, conv2d refers to a typical convolution, avgpool indicates the average pooling, c represents the amount of output networks and ‘n’ indicates the recurrent time. On the whole, the network comprises of 19 layers whereas the middle layer is used for the extraction of features. Further, the final layer is utilized for classification. According to transfer learning model, the current study initially employed ‘MobileNetV2 pre-trained model’, using ImageNet, as a feature extractor and trained two layers which are later fine-tuned with a few layers rather than each trainable layer.

2.3 Hyperparameter Optimization: SOS Algorithm

In this work, the hyperparameter involved [17–19] in MobileNetv2 model are fine-tuned using SOS algorithm [20]. Being an advanced method, SOS algorithm is used to resolve the optimization problems according to the interaction among organisms in nature. In general, organisms hardly live in isolation because each organism is dependent upon another for both survival and nutrition. These relationships mainly exist on the basis of trust. In other terms, this relationship is called symbiotic relationship. SOS approach initiates in the presence of ecosystem with initial population. At first, a collection of organisms is arbitrarily created. These arbitrarily-created organisms denote the solution for the presented problem. SOS approach yields a novel solution based on stimulation of living relations between two ecosystems. The pseudocode of the SOS algorithm includes parasitism, mutualism, and commensalism of natural biological relations.

Mutualism

In this type of relationship, both the organisms benefit from each other. For instance, assume the relationship between honeybees and flowers. The bees fly over flowers to collect the nectar needed for honey production. On the other hand, it is advantageous for the flowers since bees smatter the pollen and simplify the pollination process. During SOS process, Xi indicates the organism which is equivalent to i-th individual. The Xj second organism is chosen arbitrarily and related to Xi. In conclusion, Xi and Xj are upgraded in Mutualism stage as follows:

Xinew=Xi+rand(0,1)×(Xbest−Mutua1−Vector×BF1)(6)

Xjnew=Xj+rand(0,1)×(Xbest−Mutua1−Vector×BF2)(7)

Mutua1−Vector=(Xi+Xj)/2(8)

In the above equation, rand(0,1) denotes an arbitrary number of vectors. BF1 and BF2 indicate the profit factors of Xi and Xj that demonstrate the profit of every organism. In Eq. (6), Mutual_Vector characterizes the relationships between Xi and Xj. The Mutual_Vector∗BF2 in Eqs. (6) and (7) is the effort taken to increase the survival rate of living organisms. As per Darwinian concept, i.e., survival of the fittest, each organism has to rise its compatibility with its surroundings. Now, Xbest characterizes the maximum phase of compatibility.

Commensalism

Here, one of the organisms gains benefits without harming the other organism in the relationship. For instance, in the relationship between sharks and sticky fish, the sticky fish sticks around the shark and feed on residual foods; the shark gains small or no benefits. In this Mutualism process, the Xj organism is arbitrarily chosen and is related to Xi. Here, Xi strives to obtain the benefits while Xj gets no benefits or losses. Therefore, Xi is upgraded as follows.

Xinew=Xi+rand(−1,1)×(Xbest−Xj)(9)

In Eq. (9), Xbest−Xj indicates the benefit given when using Xj for increased survival of Xi.

Parasitism

In this relationship type, one organism gains the benefit out of the other organism while the second one faces loss. For instance, malarial blood parasites attack the human body and proliferate which, at times, cause death also. In SOS process, Xi, viz., malaria mosquito generates an artificial parasite named ‘Parasite-Vector’. It is generated in the searching region with the replication of Xi. Here, Xj is arbitrarily chosen using the ecosystem and the parasite serves as its host. Parasite-Vector tries to capture the position of Xj in the ecosystem. Both Xi and Xj are estimated to determine the capability. Once parasite-vector demolishes Xj and takes its position in the ecosystem, it accomplishes the maximum capability. The maximum capability for Xj can be accomplished once it resists the parasite, and the parasite cannot live further in the ecosystem.

:

2.4 Image Classification: GRU Model

In this final stage, GRU technique is executed as a classification model to determine suitable class labels [21]. GRU is an efficient memory cell that can be used for different kinds of applications. It is viewed as an improvement of Long Short Term Memory (LSTM) and its performance can be compared and generalized. In order to define a GRU evidently, the researchers proposed LSTM. In Recurrent Neural Network (RNN), hidden unit is a major element since it is accountable for forgetting or remembering certain datasets. LSTM, presented by Hochreiter and Schmidhuber, is an effective enforcement and has several alternatives to its credit. The connection relationship is shown in the following equation.

{ ft=σ(Wxfxt+Whfht−1+WcfCt−1)it=σ(Wxixt+Whiht−1+WciCt−1)  Ct=ft⊙Ct−1+it⊙tanh(Wxcxt+Whcht−1)            ot=σ(Wxoxt+Whoht−1+WcoCt−1)                             ht=ot⊙tanh(Ct)  (10)

In this formula, C represents the cell state, x indicates the input vector and h denotes the resultant vector. Here, t characterizes the existing time and t−1 indicates the final time. σ indicates a sigmoid function, ⊙ implies Hadamard product and W embodies an uncertain variable. f signifies the forget gate that decides what data needs to be removed from the cell state. i denotes the input gate that decides which dataset needs to be saved in the cell states. o indicates the output gate which decides the dataset that should be given to output. In contrast to LSTM, GRU involves certain simplification processes. The connection relationship is demonstrated herewith.

{ rt=σ(Wrxt+Urht−1)zt=σ(Wzxt+Uzht−1)h˜t=tanh(Whxt+U(rt⊙ht−1))           ht=(1−zt)ht−1+zth˜t(11)

From the equation given below, h symbolizes the output vector, h~ represents the candidate output and x indicates the input vector. Here, z represents the update gate and r denotes the reset gate of GRU. Hence, GRU is simple when compared to LSTM and has fewer parameters. However, it provide great benefits in terms of convergence and performance. GRU yielded high benefits in the experiments. Fig. 2 illustrates the infrastructure of GRU method.

Figure 2: Structure of GRU

3  Results and Discussion

The performance of the proposed SOSDL-ODC method was validated using a dataset [22] that comprises of 1,144 images under three classes. Table 1 shows the details of the dataset considered for the study. Some of the sample images are illustrated in Fig. 3.

Figure 3: Sample images

Fig. 4 displays a set of confusion matrices generated by the proposed SOSDL-ODC method on the applied dataset under distinct number of epochs. The figures imply that the proposed SOSDL-ODC method achieved high performance under every epoch. For samples, with 200 epochs, the proposed SOSDL-ODC method classified 310 samples under VT, 241 samples under NVT, and 513 samples under NT class. Along with that, with 400 epochs, the proposed SOSDL-ODC approach categorized 313 samples under VT, 241 samples under NVT, and 512 samples under NT classes. Followed by, with 600 epochs, SOSDL-ODC algorithm identified 327 samples under VT, 252 samples under NVT, and 525 samples under NT classes. In line with this, with 800 epochs, the proposed SOSDL-ODC system identified 327 samples under VT, 252 samples under NVT, and 526 samples under NT classes.

Figure 4: Confusion matrices generated by SOSDL-ODC technique at (a) epoch-200, (b) epoch-400, (c) epoch-600, and (d) epoch-800

Tab. 2 and Fig. 5 shows the analysis results accomplished by SOSDL-ODC method under distinct number of classes and epochs. The experimental outcomes imply that the proposed SOSDL-ODC method has the ability to attain effectual outcomes under all classes and epochs. For sample, with 200 epochs, SOSDL-ODC methodology gained average accuy, precn, recal, Fscore, and MCC values such as 95.34%, 92.71%, 92.40%, 92.55%, and 88.93% respectively. Eventually, with 400 epochs, the proposed SOSDL-ODC system reached average accuy, precn, recal, Fscore, and MCC values such as 95.45%, 92.96%, 92.63%, 92.79%, and 89.25% correspondingly. Meanwhile, with 600 epochs, the presented SOSDL-ODC methodology gained average accuy, precn, recal, Fscore, and MCC values such as 97.67%, 96.32%, 96.18%, 96.25%, and 94.45% correspondingly. At last, with 800 epochs, the proposed SOSDL-ODC system reached average accuy, precn, recal, Fscore, and MCC values such as 97.73%, 96.42%, 96.24%, 96.33%, and 94.58% correspondingly.

Figure 5: Average analysis results of SOSDL-ODC technique at (a) epoch-200, (b) epoch-400, (c) epoch-600, and (d) epoch-800

Both Training Accuracy (TA) and Validation Accuracy (VA) values, attained by the proposed SOSDL-ODC approach on test dataset, are demonstrated in Fig. 6. The experimental outcomes imply that the proposed SOSDL-ODC method gained the maximum TA and vA values. To be specific, VA is superior to TA.

Figure 6: TA and VA analyses results of SOSDL-ODC technique

Both Training Loss (TL) and Validation Loss (VL) values, achieved by SOSDL-ODC methodology on test dataset, are shown in Fig. 7. The experimental outcomes infer that the proposed SOSDL-ODC system achieved the least TL and VL values. To be specific, VL seemed to be lower than TL.

Figure 7: TL and VL analyses results of SOSDL-ODC technique

A brief precision-recall inspection was conducted upon SOSDL-ODC method on test dataset and the results are depicted in Fig. 8. By observing the figure, it can be inferred that the proposed SOSDL-ODC model accomplished high precision-recall performance under all the classes.

Figure 8: Precision-recall curve analysis results of SOSDL-ODC technique

A detailed ROC investigation was conducted upon SOSDL-ODC approach on test dataset and the results are showcased in Fig. 9. The results indicate that the proposed SOSDL-ODC algorithm exhibited its ability to categorize three different classes such as VT, NVT, and NT on test dataset. In order to validate the enhanced performance of SOSDL-ODC method, a detailed comparative analysis was conducted and the results are shown in Tab. 3 [23,24].

Figure 9: ROC curve analysis results of SOSDL-ODC technique

A detailed precn analysis was conducted between SOSDL-ODC method and other existing models and the results are shown in Fig. 10. The figure highlights that VGG16, LeNet, and ReseNt50 models achieved the least precn values such as 79.19%, 74.62%, and 77.98% respectively. On the other hand, AlexNet model attempted to achieve an increased precn value of 81.30%. Though Sequential RNN, Non-regularized CNN, and Regularized CNN models accomplished enhanced precn values such as 89.08%, 88.67%, and 90.53%, the presented SOSDL-ODC method gained the maximal precn value of 96.42%.

Figure 10: Precn analysis results of SOSDL-ODC technique and other recent methodologies

A brief recal analysis was conducted between the proposed SOSDL-ODC method and other existing algorithms and the results are portrayed in Fig. 11. The figure reveals that VGG16, LeNet, and ReseNt50 methods achieved the least recal values such as 74.48%, 67.59%, and 79.36% correspondingly. Also, AlexNet model attempted to demonstrate an enhanced recal value of 75.18%. Eventually, Sequential RNN, Non-regularized CNN, and Regularized CNN techniques accomplished enhanced recal values such as 90.49%, 92.55%, and 93.99%. However, the presented SOSDL-ODC method achieved the maximum recal value of 96.24%.

Figure 11: Recal analysis results of SOSDL-ODC technique and other recent methodologies

A detailed accuy analysis was conducted between SOSDL-ODC method and other existing models and the results are shown in Fig. 12. The figure infers that VGG16, LeNet, and ReseNt50 algorithms achieved the least accuy values such as 80.90%, 68.55%, and 76.52% correspondingly. Followed by, AlexNet model tried to achieve a superior accuy value of 73.93%. Further, Sequential RNN, Non-regularized CNN, and Regularized CNN approaches accomplished high accuy values such as 88.77%, 90.45%, and 91.52%. However, the projected SOSDL-ODC method gained the maximum accuy of 97.73%. Based on the discussions made in the above mentioned tables and figures, it is evident that the proposed SOSDL-ODC method is superior to other existing models.

Figure 12: Accuy analysis of SOSDL-ODC technique and other recent methodologies

4  Conclusion

In this work, a new SOSDL-ODC approach has been developed for recognition and classification of osteosarcoma with the help of histopathological images. The presented SOSDL-ODC approach initially applies pre-processing technique to enhance the quality of images. Then, MobileNetv2 model is applied to generate a suitable group of feature vectors and hyperparameter tuning of MobileNetv2 model is performed using SOS algorithm. Lastly, GRU technique is applied as a classification model to determine the suitable class labels. In order to validate the enhanced osteosarcoma classification performance of the proposed SOSDL-ODC technique, a comprehensive comparative study was conducted. The obtained outcomes established the supremacy of the proposed OSDL-ODC algorithm over other existing approaches. In future, an ensemble of DL-based models can be developed to significantly boost the classification results of SOSDL-ODC model.

Funding Statement: The Deanship of Scientific Research (DSR) at King Abdulaziz University (KAU), Jeddah, Saudi Arabia has funded this project, under grant no KEP-1-120-42.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.