Open Access

ARTICLE

# Predictive Multimodal Deep Learning-Based Sustainable Renewable and Non-Renewable Energy Utilization

1 Department of Information Systems, College of Business Administration in Hawtat bani Tamim, Prince Sattam bin Abdulaziz University, Howtat Bani Tamim, 16622, Saudi Arabia

2 Department of Biomedical Engineering, College of Engineering, Princess Nourah bint Abdulrahman University, P. O. Box 84428, Riyadh, 11671, Saudi Arabia

3 Department of Information Systems, College of Computer Science, King Khalid University, Abha, Saudi Arabia

4 Department of Computer Sciences, College of Computing and Information System, Umm Al-Qura University, Saudi Arabia

5 Department of Computer Science, Faculty of Computers and Information Technology, Future University in Egypt, New Cairo, 11835, Egypt

6 Department of Computer and Self Development, Preparatory Year Deanship, Prince Sattam bin Abdulaziz University, AlKharj, Saudi Arabia

* Corresponding Author: Abdelwahed Motwakel. Email:

*Computer Systems Science and Engineering* **2023**, *47*(1), 1267-1281. https://doi.org/10.32604/csse.2023.037735

**Received** 15 November 2022; **Accepted** 02 February 2023; **Issue published** 26 May 2023

## Abstract

Recently, renewable energy (RE) has become popular due to its benefits, such as being inexpensive, low-carbon, ecologically friendly, steady, and reliable. The RE sources are gradually combined with non-renewable energy (NRE) sources into electric grids to satisfy energy demands. Since energy utilization is highly related to national energy policy, energy prediction using artificial intelligence (AI) and deep learning (DL) based models can be employed for energy prediction on RE and NRE power resources. Predicting energy consumption of RE and NRE sources using effective models becomes necessary. With this motivation, this study presents a new multimodal fusion-based predictive tool for energy consumption prediction (MDLFM-ECP) of RE and NRE power sources. Actual data may influence the prediction performance of the results in prediction approaches. The proposed MDLFM-ECP technique involves pre-processing, fusion-based prediction, and hyperparameter optimization. In addition, the MDLFM-ECP technique involves the fusion of four deep learning (DL) models, namely long short-term memory (LSTM), bidirectional LSTM (Bi-LSTM), deep belief network (DBN), and gated recurrent unit (GRU). Moreover, the chaotic cat swarm optimization (CCSO) algorithm is applied to tune the hyperparameters of the DL models. The design of the CCSO algorithm for optimal hyperparameter tuning of the DL models, showing the novelty of the work. A series of simulations took place to validate the superior performance of the proposed method, and the simulation outcome emphasized the improved results of the MDLFM-ECP technique over the recent approaches with minimum overall mean absolute percentage error of 3.58%.## Keywords

Sustainable growth can be determined as retaining humanity’s sources for day-to-day requirements at a level that won’t deny future generation requirements [1]. Sustainable growth contains 3 dimensions that are environmental, economic, and social. Consequently, there has been a need to cover the increasing energy for achieving sustainable developments; in other words, to minimize the pollution of the resource utilized in this process, to enrich the quality of life, and to realize the production essential for humanity [2]. Currently, most other countries cover their energy requirements from fossil fuels like natural gas and coal, and the energy utilization of the nation surges and carbon emission is also increasing. Once fossil fuel is employed, they leave a specific number of residues from gases and solid substances. This residue, i.e., created by fossil fuels, could not be reutilized in any way; thus, it creates environmental pollution [3–5]. In that regard, sustaining the growing energy needs from renewable energy (RE) sources like biofuels, geothermal, solar, wind and biomass would assist in maintaining the pollution of the source at the lower level for sustainable growth [4].

RE is becoming a traditional preferred energy solution [6]. Wind and Solar energy could increasingly follow conventional energy sources. Simultaneously, considering that RE is low carbon, affordable, stable, trustworthy, and environmentally friendly, the consumer in developing enterprises and markets have constantly raised the requirement for RE [7]. These demand trends and driving factors are highly significant worldwide. The system absorption ability should be enhanced, and the consumption of RE should be optimized. With the evolution of the worldwide energy industry, RE is slowly replacing conventional fossil energy. This transformation implies that power plants and energy systems need to coordinate the total performance of equipment and multiple sites, enhance the performance of RE generation forecast, and enhance the energy production assets output [8]. Simultaneously, it is essential to guarantee the economy accordingly and the impacts on the stability of the electric grids. A RE resource is gradually incorporated with NRE resources into the power grid to meet energy needs [9]. The historical information of power load is noted at a certain period and an ordered collection sampled; hence they are a time series. Incorporating solar energy into the energy system is growing; therefore, forecasting solar energy production has significantly improved system reliability and control energy quality [10]. In the field of artificial intelligence (AI), deep learning models have focused on gaining more accurate and reliable systems and have shown excellent tools to resolve several energy application challenges.

This study presents a new multimodal fusion-based predictive tool for energy consumption prediction (MDLFM-ECP) of RE and NRE power sources. The proposed MDLFM-ECP technique involves MDLFM-ECP technique involves the fusion of four deep learning (DL) approaches, namely long short-term memory (LSTM), bidirectional LSTM (BiLSTM), deep belief network (DBN), and gated recurrent unit (GRU). Also, the chaotic cat swarm optimization (CCSO) algorithm is applied to tune the hyperparameters of the DL models. A wide range of experimental analyses is carried out to examine the improved outcomes of the proposed model. In short, the key contributions are summarized as follows.

• Developed a new MDLFM-ECP technique for the prediction of energy consumption under RE and NRE power sources.

• Perform a fusion-based prediction process using four DL models namely LSTM, BiLSTM, GRU, and DBN.

• Employ the CCSO algorithm as a hyperparameter optimizer to improve the predictive outcomes of the DL models.

Moradmand et al. [11] examine the energy management of this system in 3 corresponding scenarios. The 1st and 2nd scenarios are correspondingly appropriate for the markets with an inconstant and constant price. In addition, the 3rd scenario is presented for considering the uncertainty in RES, and it is proposed for recompensing the electricity shortages with NRE source. The legitimacy of the method can be examined in a residential region for 24 h, and the complete result demonstrates the presented model’s capability for reducing the operation cost. Nabavi et al. [12] predict commercial and residential energy requirements in Iran with 3 distinct ML algorithms involving logarithmic multiple linear regression methods, multiple linear regression, and non-linear autoregressive with exogenous input. This method is designed according to the various aspects involving the shares of RE resources in gross domestic production, total energy utilization, cost of natural gas, electricity, and the population.

Liu et al. [13] developed an inaccurate Bi-level optimization algorithm based on a regional-scale hybrid RE and NRE planning system. Elahi et al. [14] predicted energy use indices, the energy flow of buffalo farms, energy utilization target, production efficacy, sensitivity analysis of energy inputs impacts, and energy inputs on energy outcome. A well-structured questionnaire was utilized to collect information on 360 domestic buffalo farms from May–July 2017. The result shows that milk manufacture was based largely on millet, RE input, concentrates, sorghum, and minerals. Behera and Behera et al. [15] aim to examine the relationships between RE and NRE utilization and economic development in the G7 nations (France, the United States, the United Kingdom, Canada, Italy, and Germany). The research utilized the Pesaran CADF panel second-generation unit root testing to verify the variable’s stationary property. The research utilized the panel autoregressive distributed lag (P-ARDL) method to explore the short-run and long-run dynamics. Ikram [16] designed the incorporated grey architecture for forecasting the development trend of RE and NRE consumption and production in Asian and Oceania regions in the period of 2017–2025 through new grey prediction methods for measuring the performances of the grey model.

Zhao et al. [17] present an adjacent accumulation discrete grey method for enhancing the grey approach’s predictive performance and using novel information. The steadiness of the presented method is verified, and the novel model could attain high accuracy by using 3 cases. Also, it is employed to predict the NRE utilization in Asia-Pacific Economic Cooperation. It shows that the discrete grey approach using adjacent accumulation is efficient. Nadimi et al. [18] employ an econometrics method for forecasting the energy utilization of Japan till 2030. Next, it employs a stochastic substitution method for fitting appropriate RE. A significant part of the presented method is based on the Random Number generation and the recursive Bayesian filter to upgrade the distribution of RE systems via substitution.

This study presents a new MDLFM-ECP technique for predicting energy consumption on RE and NRE sources. The proposed MDLFM-ECP technique involves pre-processing, DL based fusion model for prediction, and CCSO-based hyperparameter optimization. Fig. 1 illustrates the overall workflow involved in the MDLFM-ECP model. The comprehensive operation of these progressions is demonstrated in the following:

3.1 Stage 1: Data Pre-Processing

In the RE and NRE sources, missing data exist due to climate, preservation, abandoned winds, sensors fault, measurement error, etc. The new information from predicting techniques adversely affects outcomes’ forecast accuracy and consistency. Therefore, the pre-treatment of historical information to wind power forecasts is attracting considerable attention. During this analysis, a 2-manner comparative technique was utilized for identifying the abnormal information because of the inability of manual work to find the wrong information from huge datasets. The 2-manner comparative technique enhances the condition in which the typical lateral comparative approach could not effectually manage the loss and mutation of continuous data from the procedure of judgment and modification of abnormal information. Also, this work accepts the succeeding formula for completing the linear transformation to 0 and 1 [10]:

where

3.2 Stage 2: Fusion-Based Prediction Process

At this stage, the pre-processed data are applied to the fusion model, which consists of four DL models. The fusion of features is an important process that integrates multiple features. It mainly depends upon the features fusion using entropy, and the four vectors can be represented using Eqs. (2)–(4):

Integrating derived features into a single vector [19] using Eq. (5).

where

GRU is a simple variant of LSTM, which consists of 2 gates, an “update gate” that has an input gate, forget gate and a “reset gate”. The GRU has no other memory cell for keeping the data. Hence, it could only control data inside the unit [20].

Now,

RNN has been used for regression problems with the consideration of temporal dependency. The unfolded RNN holds the capacity to operate present data through preliminary data. In the meantime, RNN suffers from the issue of training the long-term dependency, i.e., resolved by the variations of RNN. Therefore, the LSTM is proposed as an advanced RNN network and addresses the limitations of RNN using hidden layer units called memory cells. It is controlled by 3 gates such as output, forget, and input gates [21]. Fig. 2 depicts the architecture of the LSTM model.

The process of input, as well as output gates, is utilized for controlling the flow of memory cell input and output to the remaining networks. Additionally, forget gate was appended to the memory cell that passes the outcome data through higher weights before the following neurons. The data exist in memory based on the higher activation result; when the input unit holds higher activation, the data is saved in the memory cell. Furthermore, when the output element has higher activation later, it passes the data to the following neurons. Or else, input data with the highest weights reside in memory cells. LSTM model computes the map among output and input sequences, viz.,

In Eqs. (9)–(13)

To address the limitation of LSTM cell, i.e., capable of processing prior content but not utilising the upcoming one. It developed the BRNN method, i.e., contained 2 different LSTM hidden units with equivalent output in contradictory directions. This framework can employ prior and upcoming data in the output layer. The input series

DBN holds a bi-directional association known as RBM type connection on every peak layer while the underneath layer has only top/down association. They are trained layer-wise as pretraining. The pretraining exists by training the networking elements-wise bottom-up approach, viz., by processing the first 2 layers as an RBM, later preparing the 2nd and 3rd layers as another RBM, and eventually preparing for this parameter. Fig. 4 demonstrates the framework of the DBN model. The DBN model can be mathematically formulated by [22],

whereas

3.3 Stage 3: Hyperparameter Tuning Using CCSO Algorithm

The CCSO is a population-based metaheuristic approach stimulated by the natural characteristics of cats and consists of two sub-models: tracing mode (TM) and seeking mode (SM), that defines the nature of the cats. This technique presented the optimization problem in the continuous field, as defined below.

where

Step 1: Generate

Step 2: Arbitrarily sprinkle the cat into the

Step 3: Estimate the fitness value (FV) of all cats by implementing the places of cats into the fitness function (FF) that demonstrates the condition of our purpose and preserves the optimum cat as to memory. Noticeably, it can only require remembering the place of an optimum cat

Step 4: Shift the cats based on the flags. When

Step 5: Re-pick the number of cats by setting a trace mode based on

Step 6: Verify the end criteria. If fulfilled, end the process; else, reiterate steps 3–5. The chaotic map has been discrete-time dynamical model

The chosen chaotic map takes chaotic numbers from 0 and 1. It offers novel techniques presenting a chaotic map in CCSO to improve global convergence by absconding the local solution.

Examining cat behaviors, the 2 behaviors of cats, such as TM and SM: Cats often remain attentive and move very slowly. This behavior of the cat is denoted as SM. Once the existence of prey is recognized, the cat chases it rapidly. This behavior of a cat, viz., chasing with higher speed, is denoted as TM. The cat’s location is utilised to represent the solution set [23]. All the cats have velocity and position for every fitness value and dimension. Besides this, a flag is utilized to identify the cat in TM/SM.

In TM, the solution search (Update the location of

Whereas

Based on Eq. (18), the arbitrary number

whereas

This section investigates a comprehensive experimental validation of the MDLFM-ECP technique on the test data, and the results are examined under varying aspects. In this work, four kinds of energy sources in dataset namely fossil fuel (FF), behind the meter (BTM), wind power (WP), and power purchase agreement (PPA). Table 1 provides a detailed comparative results analysis of the MDLFM-ECP technique [24]. The proposed model is simulated using Python 3.6.5 tool on PC i5-8600k, GeForce 1050Ti 4GB, 16GB RAM, 250GB SSD, and 1TB HDD. The parameter settings are given as follows: learning rate: 0.01, dropout: 0.5, batch size: 5, epoch count: 50, and activation: ReLU.

Fig. 5 offers the MAE and RMSE results of the MDLFM-ECP and existing techniques. Observing the results in terms of MAE, the results portrayed that the SVR, Lasso, Ridge, and MLP techniques have accomplished poor outcomes with higher MAE values of 75.8, 58.7, 56.7, and 56.2, respectively. In addition, the G-Boost and XGBoost techniques have resulted in moderately closer MAE of 24.8 and 23.5, respectively. Though the ML-PEC technique has gained near optimal MAE of 15.7, the proposed MDLFM-ECP technique has showcased effective outcomes with the least MAE of 14.36. Likewise, on detecting the outcomes for RMSE, the results exhibited that the SVR, Lasso, Ridge, and MLP approaches have accomplished the least result with the maximal RMSE values of 98.8, 73.3, 71.8, and 72.5, correspondingly. Moreover, the G-Boost and XGBoost systems have correspondingly resulted in moderately closer RMSE of 33.5 and 27.9. But, the ML-PEC manner has attained near optimum RMSE of 21.7, and the projected MDLFM-ECP technique has outperformed the effective outcome with the minimum RMSE of 19.78.

Fig. 6 provides the MSE analysis of the MDLFM-ECP manner with existing techniques. The figure shows that the SVR, Lasso, Ridge, and MLP systems have accomplished poor outcomes with higher MSE values of 9753.2, 5373.7, 5151.0, and 5250.3, respectively. In addition, the G-Boost and XGBoost approaches have resulted in moderately closer MSE of 1119.6 and 777.5, respectively. Though the ML-PEC scheme has gained near optimal MSE of 473.0, the presented MDLFM-ECP technique has outperformed effective outcomes with the least MSE of 391.25.

Fig. 7 gives the RMSLE results of the MDLFM-ECP and existing approaches. The results demonstrated that the SVR, Lasso, Ridge, and MLP techniques had accomplished poor outcomes with superior RMSLE values of 0.17, 0.13, 0.13, and 0.12. Followed by the G-Boost and XGBoost manners have resulted in moderately closer RMSLE of 0.06 and 0.05, respectively. Afterwards, the ML-PEC technique gained near optimal RMSLE of 0.04, and the presented MDLFM-ECP technique has portrayed a productive outcome with a minimal RMSLE of 0.03.

Table 2 provides the results analysis of the proposed model in terms of actual and predicted error under three runs. Fig. 8 demonstrates the results analysis of the MDLFM-ECP technique in terms of actual vs. predicted error under run-1. The results demonstrated that the proposed model had attained the least differences between the actual and predicted error.

Fig. 9 showcases the performance analysis of the projected method in terms of actual vs. predicted error under run-2. The outcomes showed that the presented system had attained minimum differences between actual and predicted errors.

Fig. 10 illustrates the performance analysis of the proposed approach with respect to actual vs. predicted error under run-3. The results outperformed that the projected model has gained minimum differences between the actual and predicted error.

Finally, an overall MAPE analysis of the MDLFM-ECP technique with existing techniques takes place in Table 3 and Fig. 11. The experimental outcome has shown that the SVR, MLPRegressor, Ridge, and Lasso techniques have obtained higher MAPE of 21.61, 19.52, 10.41, and 10.39 respectively. In line with this, the XGBoost, Gradient Boosting, and ML-PEC-RE + NRE techniques have obtained moderately closer MAPE of 5.77, 6.86, and 4.29, respectively. However, the presented MDLFM-DCP technique has outperformed the other techniques with a MAPE of 3.58.

The results mentioned above ensure that the MDLFM-ECP technique has accomplished proficient outcomes with the minimum MAE, MSE, RMSE, RMSLE, and MAPE of 14.36, 391.25, 19.78, 0.03, and 3.58, respectively. Therefore, the proposed MDLFM-ECP technique is an effective tool for predicting energy utilization.

This study presents a new MDLFM-ECP technique for predicting energy utilization on RE and NRE sources. Actual data may influence the prediction performance of the results in prediction approaches. The proposed MDLFM-ECP technique involves pre-processing, DL based fusion model for prediction, and CCSO-based hyperparameter optimization. Moreover, the CCSO algorithm is applied to tune the hyperparameters of the DL algorithms. A series of simulations take place to ensure the superior efficiency of the MDLFM-ECP technique, and the simulation outcome emphasizes the improved results of the MDLFM-ECP technique over the other approaches. As a part of the future scope, the presented MDLFM-ECP technique can be tested using real-time data.

Funding Statement: The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work through the Large Groups Project under grant number (71/43). Princess Nourah bint Abdulrahman University Researchers Supporting Project number (PNURSP2023R203), Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia. The authors would like to thank the Deanship of Scientific Research at Umm Al-Qura University for supporting this work by Grant Code: 22UQU4340237DSR61. This study is supported via funding from Prince Sattam bin Abdulaziz University project number (PSAU/2023/R/1444).

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

## References

- T. Güney, “Renewable energy, non-renewable energy and sustainable development,” International Journal of Sustainable Development & World Ecology, vol. 26, no. 5, pp. 389–397, 2019.
- I. Ozturk and A. Acaravci, “Electricity consumption and real GDP causality nexus: Evidence from ARDL bounds testing approach for 11 MENA countries,” Applied Energy, vol. 8, no. 8, pp. 2885–2892, 2021.
- M. B. Jebli, S. B. Youssef and I. Ozturk, “Testing environmental Kuznets curve hypothesis: The role of renewable and non-renewable energy consumption and trade in OECD countries,” Ecological Indicators, vol. 60, pp. 824–831, 2011.
- N. Apergis and D. C. Danuletiu, “Renewable energy and economic growth: Evidence from the sign of panel long-run causality,” International Journal of Energy Economics and Policy, vol. 4, no. 4, pp. 578–587, 201
- A. E. Abouelregal and M. Marin, “The response of nanobeams with temperature-dependent properties using state-space method via modified couple stress theory,” Symmetry, vol. 12, no. 8, pp. 1276, 2020.
- L. Malka, A. Daci, A. Kuriqi, P. Bartocci and E. Rrapaj, “Energy storage benefits assessment using multiple-choice criteria: The case of drini river cascade,” Albania Energies, vol. 15, no. 11, pp. 4032, 2022.
- N. Suwal, X. Huang, A. Kuriqi, Y. Chen, K. P. Pandey et al., “Optimisation of cascade reservoir operation considering environmental flows for different environmental management classes,” Renewable Energy, vol. 158, no. 4, pp. 453–464, 2020.
- M. Pérez-Ortiz, S. Jiménez-Fernández, P. A. Gutiérrez, E. Alexandre, C. Hervás-Martínez et al., “A review of classification problem, algorithms in renewable energy applications,” Energies, vol. 9, no. 8, pp. 607, 2016.
- M. L. Scutaru, S. Vlase, M. Marin and A. Modrea, “New analytical method based on dynamic response of planar mechanical elastic systems,” Bound Value Probability, vol. 2020, no. 1, pp. 104, 2020.
- L. Munkhdalai, T. Munkhdalai, K. H. Park, H. G. Lee, M. Li et al., “Mixture of activation functions with extended min-max normalization for forex market prediction,” IEEE Access, vol. 7, pp. 183680–183691, 2019.
- A. Moradmand, M. Dorostian and B. Shafai, “Energy scheduling for residential distributed energy resources with uncertainties using model-based predictive control,” International Journal of Electrical Power & Energy Systems, vol. 132, no. 4, pp. 107074, 2021.
- S. A. Nabavi, A. Aslani, M. A. Zaidan, M. Zandi, S. Mohammadi et al., “Machine learning modeling for energy consumption of residential and commercial sectors,” Energies, vol. 13, no. 19, pp. 5171, 2020.
- Y. Liu, L. He and J. Shen, “Optimization-based provincial hybrid renewable and non-renewable energy planning-A case study of Shanxi, China,” Energy, vol. 128, no. 6095, pp. 839–856, 2017.
- E. Elahi, C. Weijun, S. K. Jha and H. Zhang, “Estimation of realistic renewable and non-renewable energy use targets for livestock production systems utilising an artificial neural network method: A step towards livestock sustainability,” Energy, vol. 183, no. 1, pp. 191–204, 2019.
- J. Behera and A. K. Mishra, “Renewable and non-renewable energy consumption and economic growth in G7 countries: Evidence from panel autoregressive distributed lag (P-ARDL) model,” International Economics and Economic Policy, vol. 17, no. 1, pp. 241–258, 2020.
- M. Ikram, “Models for predicting non-renewable energy competing with renewable source for sustainable energy development: Case of asia and oceania region,” Global Journal of Flexible Systems Management, vol. 22, no. 2, pp. 133–160, 2021.
- H. Zhao and W. Lifeng, “Forecasting the non-renewable energy consumption by an adjacent accumulation grey model,” Journal of Cleaner Production, vol. 275, no. 7, pp. 124113, 2020.
- R. Nadimi and K. Tokimatsu, “Analyzing of renewable and non-renewable energy consumption via bayesian inference,” Energy Procedia, vol. 142, pp. 2773–2778, 2017.
- T. Saba, A. S. Mohamed, M. El-Affendi, J. Amin, M. Sharif et al., “Brain tumor detection using fusion of hand crafted and deep learning features,” Cognitive Systems Research, vol. 59, no. 1, pp. 221–230, 2020.
- R. Dey and F. M. Salem, “Gate-variants of gated recurrent unit (GRU) neural networks,” in IEEE 60th Int. Midwest Symp. on Circuits and Systems (MWSCAS), Boston, MA, USA, pp. 1597–1600, 2017.
- F. Shahid, A. Zameer and M. Muneeb, “Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM,” Chaos Solitons & Fractals, vol. 140, no. 14, pp. 110212, 2020.
- K. Mathivanan, D. Thirumalaikumarasamy, M. Ashokkumar, S. Deepak and M. Mathanbabu, “Optimization and prediction of az91d stellite-6 coated magnesium alloy using box behnken design and hybrid deep belief network,” Journal of Materials Research and Technology, vol. 15, no. 1, pp. 2953–2969, 2021.
- D. Gabi, N. M. Dankolo, A. S. Ismail, A. Zainal, Z. Zakaria et al., “Non-preemptive chaotic cat swarm optimization scheme for task scheduling on cloud computing environment,” International Journal of Advanced Computer Research, vol. 9, no. 43, pp. 186–196, 2019.
- P. W. Khan, Y. C. Byun, S. J. Lee, D. H. Kang, J. Y. Kang et al., “Machine learning-based approach to predict energy consumption of renewable and non-renewable power sources,” Energies, vol. 13, no. 18, pp. 4870, 2020.

## Cite This Article

A. Motwakel, M. Obayya, N. Nemri, K. Tarmissi, H. Mohsen*et al.*, "Predictive multimodal deep learning-based sustainable renewable and non-renewable energy utilization,"

*Computer Systems Science and Engineering*, vol. 47, no.1, pp. 1267–1281, 2023.