As a result of the increased number of COVID-19 cases, Ensemble Machine Learning (EML) would be an effective tool for combatting this pandemic outbreak. An ensemble of classifiers can improve the performance of single machine learning (ML) classifiers, especially stacking-based ensemble learning. Stacking utilizes heterogeneous-base learners trained in parallel and combines their predictions using a meta-model to determine the final prediction results. However, building an ensemble often causes the model performance to decrease due to the increasing number of learners that are not being properly selected. Therefore, the goal of this paper is to develop and evaluate a generic, data-independent predictive method using stacked-based ensemble learning (GA-Stacking) optimized by a Genetic Algorithm (GA) for outbreak prediction and health decision aided processes. GA-Stacking utilizes five well-known classifiers, including Decision Tree (DT), Random Forest (RF), RIGID regression, Least Absolute Shrinkage and Selection Operator (LASSO), and eXtreme Gradient Boosting (XGBoost), at its first level. It also introduces GA to identify comparisons to forecast the number, combination, and trust of these base classifiers based on the Mean Squared Error (MSE) as a fitness function. At the second level of the stacked ensemble model, a Linear Regression (LR) classifier is used to produce the final prediction. The performance of the model was evaluated using a publicly available dataset from the Center for Systems Science and Engineering, Johns Hopkins University, which consisted of 10,722 data samples. The experimental results indicated that the GA-Stacking model achieved outstanding performance with an overall accuracy of 99.99% for the three selected countries. Furthermore, the proposed model achieved good performance when compared with existing bagging-based approaches. The proposed model can be used to predict the pandemic outbreak correctly and may be applied as a generic data-independent model to predict the epidemic trend for other countries when comparing preventive and control measures.
In recent years, machine learning (ML) has been utilized as a method to forecast and predict COVID-19 cases [
The field of ML has paid significant attention to Ensemble Machine Learning (EML) techniques, and these methods have shown significant predictive power and stability in numerous applications [
The stacking process involves different types of base learners and meta-learners; base learners are trained on all available data in the sample. However, a meta-learner is trained on the predictions made by base learners to produce the final prediction [
The proposed stacking ensemble-based model will utilize the ability of Genetic Programming (GP) to explore and exploit the classifier search space effectively. To the authors’ knowledge, this is the first study to determine the best way to select the optimal combination of base classifiers and meta-classifiers using genetic algorithms. The contribution can be summarized as follows. A hybrid stacking ensemble learning GA-Stacking model is proposed to predict the number of new COVID-19 cases. In this model, DT, Least Absolute Shrinkage and Selection Operator (LASSO), Random Forest (RF), RIGID regression and eXtreme Gradient Boosting (XGBoost) are used as the first-level base learners, and LR is used as a meta-learner (the second level). A new algorithm is developed that utilizes GA to obtain the best number and combination of base classifiers/learners for the ensemble prediction model. Furthermore, the model hyperparameters are optimized and the optimization process is verified through experiments to improve the model’s identification ability. Extensive numerical experiments are conducted on the collected data to assess the effectiveness of the proposed method. The results demonstrate that the proposed method outperforms popular traditional machine learning methods, including DT, RF, LASSO regression, RIGID regression, and XGBoost. Moreover, additional experiments are conducted to study the impact of vaccine indicators on the model performance for predicting new, positive COVID-19 cases. Unlike the state-of-the-art methods that fail to achieve a cross dataset generalization for a variety of countries with similar preventive and control measures (such as quarantine policies), the proposed adaptive model yields adequate performance under cross-dataset environments for different confirmed COVID-19 cases in different countries.
To combat COVID-19, Deep Learning (DL), Artificial Intelligence (AI), and ML can be used for different applications, such as future likely case prediction, drug development, vaccines, and early disease monitoring and tracking [
In [
Despite the great ability of such single predictive models to mitigate some of the uncertainty associated with the progression of COVID-19, the ensemble models tend to outperform such models. Ensemble learning mainly involves training base classifiers and formulating combinations of base classifiers. The three most popular ensemble strategies are bagging, boosting, and stacking [
In [
Three popular classifiers, extra trees, RF, and logistic regression, were used in [
Similarly, the authors in [
Ensemble training based on genetic algorithms was utilized in [
In [
Although recent research, including AI-based and ML-based models for COVID-19 predictions, has received more attention and promising initial results have been obtained, as shown in Some techniques need multiple steps to offer reliable model design, and the relative importance of the employed classifiers cannot be obtained directly by using most of the corresponding stand-alone approaches. It may be difficult to attain an optimal structure for models when several features are present; developing models can require significant computational resources and expertise. Hybrid ensemble-based models, which combine several predictive models, help improve the accuracy of predictions. However, building an ensemble often causes the performance of the model to decrease due to the increasing number of learners that are not being properly selected. Most hybrid COVID-19 trend prediction models use the output of only one model as the input features for another model or use a voting mechanism. Predictive performance would be improved through the development of an automatic learning model from a multitude of different predictive models. The deep learning-based COVID-19 prediction models are sensitive to the initial values of model parameters, such as the number of hidden layers, training features, and model hyperparameters. The determination of initial values of these parameters is usually based on domain knowledge, and any update for AI models to reflect the changing dynamics of the pandemic (such as an increasing number of confirmed new cases) is challenging because it requires more training time. To perform early and targeted therapies for new COVID-19 cases, researchers have conducted studies combining statistical learning and artificial intelligence algorithms. It is not only the accuracy of the classifier that affects the performance of an ensemble but also the diversity of the classifiers used in the ensemble.
[ |
Real-time COVID-19 forecasting, including incidence and cumulative weekly deaths and confirmed cases | Rate of daily hospitalizations. |
DeepCOVID, a static prediction DL-based prediction and explainability module | The proposed model was used in CDC COVID-19 Forecast Hub (since April |
[ |
Predict epidemic trends within 7 days based on confirmed cases | Coronavirus Update (Live): ( |
ALeRT-COVID, a static attention-based RNN architecture using the transfer learning method | This model obtained a higher prediction in terms of future confirmed cases |
[ |
Estimate the risk of developing critical illness for patients with COVID-19 | Seventy-two potential predictors were considered from 1,590 patients with COVID-19 in the 575 hospitals of 31 provincial administrative regions in China as of January 31, 2020 | LASSO and Logistic Regression (LR) models | AUC = 0.88 (95% CI, 0.84–0.93) in a validation cohort of 710 patients |
[ |
Predict the spread of COVID-19 risk for countries | A total of 531 counties with 11 data attributes for each country over 246 days | A static DL model based on LSTM to predict the accumulated number of COVID-19 cases in the next two weeks | This consisted of 87% of the counties across the United States. A lower correlation was reported for the counties with a total number of cases of < 1000 during the test interval |
[ |
Diagnose COVID-19 Lung CT images | A total of 5,372 patients from seven cities or provinces in China. | A fully automatic DL model (DenseNet121-FPN) | AUC 0.87 and 0.88 for two validation sets in distinguishing COVID-19 from other pneumonia and AUC 0.86 in distinguishing COVID-19 from viral pneumonia |
[ |
A CNN-based ensemble model for exoplanet detection | A total of 5067 samples with 3198 features and 570 samples with 3198 features | A static CNN- ensemble-based model with six ML models, such as logistic regression, support vector machine, decision tree, multilayer perceptron, random forest, and CNN models | Accuracy = 99.62% |
[ |
Predict the COVID-19 outbreak prediction for 5 countries over 150 days | A dataset for Italy, China, Iran, Germany, and the USA with no details specified | Perform a comparative analysis of ML and soft computing models for long-term forecasting of COVID-19 | Root Mean Squared Error (RMSE) of 592.486, 1135.124, 307.585 118.247, 364.875 790.163, 364.875 and 22.354 for, linear |
[ |
Predict COVID-19 daily confirmed cases for three scenarios | Three experimental scenarios: Tunisia case study, China case study, and a third study based on China data | A static stacking-ensemble model with three learners for training, DNNs, LSTMs, and CNNs, and then the prediction results are used for meta-learner training | Accuracy of 99.97 for double stacked-DNN with 2,4736 RMSE |
[ |
Prediction of COVID-19 confirmed cases and deaths | The time-series dataset of the confirmed cases and deaths related to COVID-19 in 12 states of Mexico (and information about the whole country) | Genetic Optimization of Ensemble Neural Network Architectures for COVID-19 time series prediction | Final error from 2.62 to 12190.48 for the confirmed cases. Final error from 0.11 to 11.09 for deaths in the selected countries |
[ |
Predict COVID-19 and forecast the number of confirmed cases anddeathsin the USA, India, and Brazil | The dataset for COVID-19 is taken from the official website of the World Health Organization | CoBiD-Net ensemble by combining Bidirectional LSTM and Convolutional LSTM | The accuracy of COVID-19 cases ranges from 98.10% to 99.13% with MAPE ranging from 0.87 to 1.90 |
The objective of this research is to develop a novel stacking-based ensemble learning model based on genetic methods to aid country planners and government agencies in identifying and predicting pandemics more effectively and efficiently. This model addresses the following issues:
Developing a stacking ensemble strategy to mitigate high classification accuracy. Identifying the best numbers of base classifiers. Identifying the best combination of base classifiers that can further enhance the model performance. Increasing the model generalization ability for the samples using unbalanced data. Optimizing the model parameters to achieve the best performance.
In the next subsections, the authors describe the methodology used in addressing these issues using a new GA-Stacking model.
The proposed research framework is shown in
Additionally, to enhance the model fitting ability, the prediction results of base learners will be integrated with Linear Regression (LR) as a meta-learner. An important aspect concerns how to select the optimal number and combination of the base learner to train the meta-classifier and trust the combination. Therefore, a genetic-based learner selection algorithm was developed to efficiently accelerate the selection of base learner combinations and enhance the fitting ability of the stacking-based ensemble learning model. Finally, all of the classification results obtained from the constructed model are combined and evaluated using different performance evaluation metrics. In the following subsections, the details related to each model part are represented. Five commonly used evaluation indicators were used to verify the generalizability of the proposed model (mean absolute error, mean square error, mean squared log error, r-squared value, and median absolute error).
COVID-19 dataset 1 was used to monitor the worldwide spread of the pandemic [
Column | Description | Column | Description |
---|---|---|---|
iso_code | This attribute represents a three-letter alpha 3 code assigned to each country | deaths_smoothed | This attribute denotes the number of new deaths attributed to COVID-19 (7-day smoothed) |
new_cases_smooth | This attribute denotes the number of new confirmed cases of COVID-19 (7-day smoothed) | new_deaths_smooth | This attribute corresponds to the number of new deaths attributed to COVID-19 (7-day smoothed) per 1,000,000 people |
cases_smooth | This attribute is used to represent the number of new confirmed cases of COVID-19 (7-day smoothed) per 1,000,000 people | new_deaths | This attribute represents the number of new deaths attributed to COVID-19 |
cases_per_million | This attribute represents the number of new confirmed cases of COVID-19 per 1,000,000 people | deaths_per_million | This attribute shows the number of new deaths attributed to COVID-19 per 1,000,000 people |
Selecting the most relevant features to the target variable (
Stacking is a very effective ensemble learning method that is similar to bagging and boosting. The functionality of the stacking-based ensemble learning framework is explained by the following case study with ten hypothetical samples (N = 10). As shown in
Sample | First Level | Second-level Genetic-LR | Actual Label | ||
---|---|---|---|---|---|
L1 | L2 | L3 | |||
S1 | G1 | B1 | X1 | A1 | |
S2 | G2 | B2 | X2 | A2 | |
S3 | G3 | B3 | X3 | A3 | |
S4 | G4 | B4 | X4 | A4 | |
S5 | G5 | B5 | X5 | A5 | |
S6 | G6 | B6 | X6 | A6 | |
S7 | G7 | B7 | X7 | A7 | |
S8 | G8 | B8 | X8 | A8 | |
S9 | G9 | B9 | X9 | A9 | |
The second step in stack-based ensemble learning is to train the second-level model with the previous classification results The stacking ensemble model fully utilized the training data, which increased the models’ capabilities to classify new samples. When training the first-level models, the stacking ensemble learning method used the cross-validation method, which made the trained model more robust. A mix of tree-based and regression classifiers can be selected for construction of the model base learner, which is consistent with the imbalanced sample modeling problem treated in this study. The auto-selection of the optimal combination of a heterogeneous base learner can improve the classification and generalization ability of the model.
In the proposed GA-Stacking model, the first level was formed by using different algorithms to generate N base learners. The GA, which is a type of soft computing, is a powerful evolutionary approach that has been applied in different applications. The GA utilizes natural selection and recombination under defined fitness criteria. The GA was chosen based on the findings of the optimization research area, which ensures that it has excellent performance, is light on computation and is easy to use [
A series of experiments with different classification algorithms was conducted to identify the best learners to be used in the framework. A total of five classifiers were selected due to their ability to generate better classification accuracy, including DT, RF, LASSO, RIGID, and XGBoost. Additionally, GA is utilized to define the heterogeneous integration of those five base learners to enhance the generalization ability of the framework; furthermore, the fitting ability of the model is improved by integrating the prediction results with meta-learners. Additionally, to reduce biases and improve model accuracy, the data predicted by the first-level learner are used to feed the second level in the model. Hence, predictions derived from the base learners and the original dataset are employed effectively to reduce biases and improve accuracy. In the next subsections, the different classification algorithms used in the ensemble model of the proposed GA-stacking model are introduced, including boosting DTs, XGBoost, LASSO, RIGID, and RF.
DTs have been one of the most successful classifiers in recent studies, especially in CADx and eHealth. Moreover, the nature of the COVID-19 dataset used in the current study is a major factor for using DTs as the metric for calculating branch purity and results classification. While building a DT, all the input feature space is divided into smaller regions, and then it recursively categorizes each one independently [
An RF [
The subsample regression trees
Recently, the applications of LASSO as a feature classification technique have become more common among researchers with promising results [
Gradient-Based Decision Tree (GBDT) [
Suppose DT h(x) has J leaf nodes. The training sample set corresponding to the jth (j = 1, 2…, J) leaf node of the
The RIGID learning algorithm was included in the study because it has been proven to be versatile and capable of high classification power. The RIGID regression algorithm proposed by [
In ensemble learning, stacking methods have been successfully applied to various fields in different ways [
Afterward, any out-of-fold prediction generated from the initial training process can be adjusted by the second base-learner training process. Finally, the results X^ i, X^ i…..X^ i from m using m base predictors, i.e., X^ j = Ci(Xi), are stacked and optimized to find the final prediction results. Therefore, for a query instance xq from the test data D Test, the predictions from the combination of base classifiers using the meta-classifiers was considered to obtain the final 348 results. Contrary to other ensemble learning models, the predictions made by the proposed model conducted by the first-level model are used as the inputs for the second-level in the model. This effectively uses the predictions of the base learners and original dataset to increase accuracy and decrease biases. There are many types of learners, each of which has different capabilities that are reflected in the output they produce. Subsequently, it is impossible to predict which learner combination and number from unseen data will outperform the others. Therefore, the genetic algorithm GA was initialized to select the optimal number and the best combination of the base learner. GA [
GAs have unique properties that make them suitable for solving complex problems. These properties include nonlinearity, global optimization, and combinatorial optimization. Moreover, in comparison with traditional soft computing methods [
In this study, a new algorithm that develops ideas from the genetic algorithm to apply to stacking ensemble configurations is proposed. This algorithm generates a better classification performance in terms of the Mean Squared Error (MSE) measure than the average performance of the state-of-the-art classification networks. The genetic ensemble first encodes the connection between initial base learners. Then, a combination of 363 candidate classifiers with the best fitness score performance is selected by the algorithm. The offspring of the selected classifiers are then created through a crossover process. Finally, a random selection of candidate learners is applied to the mutation process. For any genetics-based condition, the solution must be mapped into the search space of the algorithm. Additionally, for solutions to successfully be described as chromosome sequences of genes, one must find a suitable representation scheme. The genes might be expressed as binary numbers, floating numbers, or alphabets. For the proposed algorithm, learner combination is encoded using binary representation. Hence, for GA training, a new metadata of N sample points is constructed. St = X^ (n) t(n) at the meta-level training, an m-dimensional feature vector is formed, and the final predictions X^ m = X^ i, X^ i……X^ i} are obtained. In this way, the GA maps prediction vectors X^ (n) of base-level predictors to target labels t(n), represented as FΘ(X^), and is developed and used to build the final model classifiers. The conceptual model of GA-Stacking is illustrated in
Adjusting the number and combination of base learners involves reducing the number of operations considered between any nodes from a given generation Ω with time complexity O(n). This is done by selecting the most accurate prediction result based on the fitness function (RMSE) for all learners’ C search space in each generation α, where C represents the number of learners. For example, in
To evaluate the performance of the proposed approach for forecasting the number of daily COVID-19 confirmed cases, three case studies were chosen: Russia (RUS), Europe (OWID_EUR) and Brazil (BRA). The goal is to accurately predict new cases of COVID-19 for each country. In fact, the countries are grouped according to the danger level, i.e., the accumulative number of new_deaths for each country. The country with a higher number of new_deaths is considered to be at a higher danger level. These three countries have been selected due to their high danger level.
For the implementation of the study, Anaconda notebook and Python as a programming language were used. The model with the highest predictive accuracy is selected out of all models. To tune the hyperparameters of the five base learners, a grid search process combined with five cross-validations is used to identify the best parameter values for each country individually.
ALGORITHM | PARAMETER |
---|---|
XGB | (OWID_EUR): max_depth = 4, subsample = 0.6, (RUS) max_depth = 4, subsample: 0.7 (BRA) max_depth = 4, subsample= 0.7 |
DT | (OWID_EUR):min_samples_split:3,min_samples_lea f :1, max_depth:10, (RUS): min_samples_split:3, min_samples_lea f:2, max_depth:20, (BRA): min_samples_split = 3, min_samples_lea f = 2, max_depth = 10 |
RF | (OWIDEUR): estimators = 170, max_depth = 10, (RUS): n_estimators = 105, max_depth = 20,(BRA): n_estimators = 100, max_depth= 17 |
LASSO | (OWID_EUR): alpha:1.7,(RUS): alpha:0.25,(BRA):alpha:0.8 |
RIGID | (OWID_EUR): alpha:0.7,(RUS): alpha:0.1,(BRA): alpha:0.5 |
In GA, the following parameters are used: retain = 0.7, random_select = 0.1, and mutate_chance = 0.1; where retain represents the percentage of the population to be retained after each generation, random_select indicates a probability of a rejected network remaining in the population, and mutate_chance indicates thes probability of a randomly mutated network.
In this subsection, experiments are conducted using default settings of parameters for each base learner. The aim is to show that optimizing the hyperparameters of base learners affects the performance of the stacked ensemble model.
ALGORITHM | MAE | RMSE | MSE | R2 |
---|---|---|---|---|
OWID | 8.9 | 9.5 | 90.9 | 0.999999 |
BRA | 2.4 | 3.0 | 9.4 | 0.999999 |
RUS | 1.8 | 1.9 | 3.5 | 0.999998 |
(a) |
||||
438.8 | 561.7 | 315539.2 | 0.997331 | |
555.3 | 635.9 | 404338.1 | 0.996579 | |
715.2 | 1004.3 | 1008677.7 | 0.991467 | |
10.8 | 12.6 | 159.4 | 0.999998 | |
719.9 | 905.9 | 820817.0 | 0.993056 | |
2.9 | 3.7 | 13.9 | 0.999999 | |
(b) |
||||
52.2 | 61.9 | 3843.5 | 0.998523 | |
135.3 | 190.1 | 36137.4 | 0.986117 | |
376.9 | 417.2 | 174075.9 | 0.933127 | |
15.8 | 17.9 | 321.2 | 0.999876 | |
423.9 | 492.7 | 242733.6 | 0.906752 | |
1.0 | 1.1 | 1.2 | 0.999999 | |
77.3 | 94.2 | 8873.1 | 0.999970 | |
227.9 | 268.7 | 72189.1 | 0.999757 | |
323 | 464 | 215311 | 0.999277 | |
3.7 | 4.2 | 18.1 | 0.999999 | |
202.1 | 261.3 | 68275.3 | 0.999771 | |
1.7 | 2.3 | 5.2 | 0.999999 | |
This subsection shows the effect of tuning the hyperparameters of the base classifiers on the results of single classifiers and the GA-Stacking approach.
Using GA for optimizing the best combination of base learners in a stacked ensemble overcomes the performance of the individual classifiers. Specifically, for RUS countries, GA-Stacking achieves 1.1 RMSE, while the best RMSE obtained by a single classifier reaches 17.9%. The best network for RUS includes three base learners (DT, LASSO, RIGID) and LR as a meta-leaner. The best one for BRA consists of five base learners (RF, DT, XGB, LASSO, RIGID) and the LR meta-leaner. From the conducted experiments, the best combination of base learners for each country is different. Therefore, using the proposed approach to automatically select the best network is more effective than state-of-the-art approaches.
DAY | RUS | OWID_EUR | BRA | |||
---|---|---|---|---|---|---|
20065 | 20063.5 | 51121 | 51114.7 | 79277 | 79279 | |
21312 | 21309.7 | 49299 | 49293.9 | 64134 | 64136 | |
20169 | 20170 | 42642 | 42645.5 | 733704 | 33708 | |
21258 | 21258.7 | 61480 | 61481.6 | 27804 | 27808 | |
20217 | 20219.9 | 60736 | 60728.7 | 64903 | 64904 | |
20633 | 20636.2 | 68084 | 68081.1 | 43836 | 43835 | |
23128 | 23128.2 | 78350 | 78350.3 | 65163 | 65163 | |
22791 | 22792.7 | 76997 | 76997.3 | 65165 | 65165 | |
24003 | 24003.2 | 62740 | 62739.8 | 54556 | 54556 | |
24693 | 24693.8 | 60331 | 60333.1 | 27783 | 27785 |
The aim of this experiment is to examine an augmented expectation by constructing stacked models determined from the RUS case study models to foresee the spread of COVID-19 in 50 other countries. All data time series for RUS are used for the training set GA-Stacking model, and the data from each country are used as a testing set. The results of 35 countries are presented in
COUNTRY | RMSE | MAE | COUNTRY | RMSE | MAE |
---|---|---|---|---|---|
FSM | 0.8124 | 0.8124 | WSM | 0.8124 | 0.8124 |
SLB | 0.8124 | 0.8124 | TZA | 0.8124 | 0.8124 |
VUT | 0.8124 | 0.8124 | VAT | 0.8124 | 0.8124 |
TCD | 1.597739 | 1.575272 | CYM | 1.82996 | 1.82996 |
GRD | 1.82996 | 1.82996 | OWID_INT | 3.591237 | 2.889018 |
BRA | 4.57457 | 4.036053 | POL | 713.03548 | 12.35455 |
SWE | 32.38616 | 19.85292 | CHE | 43.77917 | 33.26572 |
ROU | 103.0007 | 100.3612 | SVK | 104.7635 | 98.01613 |
ITA | 112.3686 | 105.2246 | MKD | 113.5128 | 104.0356 |
CHN | 130.507 | 126.2616 | SRB | 157.5344 | 156.1569 |
LTU | 207.5697 | 194.0088 | SVN | 227.1699 | 201.9805 |
MEX | 247.5803 | 233.9793 | CAF | 295.1155 | 94.45863 |
DJI | 348.304 | 221.0333 | ALB | 405.5124 | 329.0025 |
MLI | 438.9535 | 375.5951 | UKR | 440.0188 | 430.3709 |
SDN | 457.1174 | 237.7952 | VCT | 474.401 | 317.8191 |
STP | 476.6715 | 235.239 | HUN | 487.2134 | 325.3511 |
GNQ | 498.3691 | 305.2593 | ISR | 512.6972 | 500.4243 |
BIH | 514.3949 | 329.2899 |
The hybrid ensemble-based model presented in [
This section presents a comparison of the performance of the proposed GA-Stacking approach that selects base learners automatically based on a genetics algorithm to recent related work that manually selects base learners of the ensemble model presented in [
40.56 | 24.89 | |
5634.91 | 4526.65 | |
31282.92 | 15237.92 | |
3530.2 | 2110.7 | |
39628.34 | 28768.47 |
From
This section aims to investigate the impact of adding vaccine indicators to the learning features of the general GA-Stacking model. The experiments are conducted using the same setting for forecasting the new_cases, with additional vaccine features, which are presented in
COLUMN | DESCRIPTION | COLUMN | DESCRIPTION |
---|---|---|---|
ISO_CODE | This attribute represents a unique number assigned to each country | vaccinations_ |
This attribute corresponds to the new COVID-19 vaccination doses administered (7-day smoothed) |
TOTAL_ |
This attribute denotes the total COVID-19 vaccination doses administered | total_ |
This attribute represents the number of COVID-19 vaccination doses administered per 100 people |
PEOPLE_ |
This attribute is used to represent the number of people who received at least one vaccine dose | people_ |
It shows the number of people who received at least one vaccine dose per 100 people |
FULLY_ |
This attribute represents the total number of people who received all doses | vaccinations_ |
It shows the number of people who received all doses prescribed by the vaccination protocol per 100 people |
NEW_ |
This attribute stands for new COVID-19 vaccination doses administered | vaccinations_ |
It shows COVID-19 vaccination doses administered (7- day smoothed) per 1,000,000 people |
In this part, the countries arranged by their cumulative new cases were selected as an indicator to visualize the performance of the GA-Stacking general model and its relation to the spread of the epidemic in the selected countries. As can be seen in
To show the effect of adding vaccine indicators to the general model performance, the general model without (presented in
In this research, a GA-Stacking model is presented, which can be used as an accurate decision support tool to improve COVID-19 surveillance, infection control, and epidemic forecast management. Recently, most hybrid ensemble-based models have been deterministic and static, which means that all the model learners and combinations were known beforehand. Unfortunately, in real-world problems, determining all model details about the problem to be analyzed is not known at the outset. As a result, the ensemble of learners obtained by static ML can become invalid if unexpected situations occur, such as changing research area dataset types. Therefore, design methods must be used in order to build a general ensemble-based model automatically which can quickly react to changes in the search space or area under study, or prediction methods can be used so that there is the possibility of correcting it.
The proposed model employs GA-Stacking to select the best number and combination of stacking learners. The evaluation of the proposed method was conducted in three countries to measure the classification accuracy of the proposed model with and without hyperparameter optimization. The results show that optimization of base learners’ hyperparameters using the grid-search method improves the performance of both individual classifiers and the proposed GA-Stacking approach. Additionally, initializing GA for selecting the best base learners’ combination achieves high performance compared with individual classifiers. The proposed Stacked-GA model can be generalized for use in forecasting daily COVID-19 new cases in different countries. It was found that the general model works better for countries with limited and widespread COVID-19 than countries with a medium number of confirmed cases in terms of RMSE and MAE. The general GA-Stacking model was examined after adding vaccine indicators, and it has different impacts on the results from different countries. From the experiments conducted, it is recommended that vaccine features should be considered when building a forecasting model if precautionary measures are considered, such as sufficient social distancing and isolation measures. Otherwise, vaccine features will decrease the model performance, so they are not useful for accurate forecasting of daily COVID-19 spread. For future work, the model could be tested considering the GA as a meta-learner using a different number of machine learning models and artificial intelligence techniques. Moreover, different soft computing models for the long-term forecasting of COVID-19 could be proposed.