Open Access

ARTICLE

AI-Driven Malware Detection with VGG Feature Extraction and Artificial Rabbits Optimized Random Forest Model

Brij B. Gupta^1,2,3,4,*, Akshat Gaurav⁵, Wadee Alhalabi⁶, Varsha Arya^7,8, Shavi Bansal^9,10, Ching-Hsien Hsu¹

1 Department of Computer Science and Information Engineering, Asia University, Taichung, 413, Taiwan
2 Department of Medical Research, China Medical University Hospital, China Medical University, Taichung, 40447, Taiwan
3 Symbiosis Centre for Information Technology (SCIT), Symbiosis International University, Pune, 412115, India
4 School of Cybersecurity, Korea University, Seoul, 02841, Republic of Korea
5 Computer Engineering, Ronin Institute, Montclair, NJ 07043, USA
6 Department of Computer Science, Immersive Virtual Reality Research Group, King Abdulaziz University, Jeddah, 21589, Saudi Arabia
7 Hong Kong Metropolitan University, Hong Kong SAR, China
8 Center for Interdisciplinary Research, University of Petroleum and Energy Studies, Dehradun, 248007, India
9 Department of Research and Innovation, Insights2Techinfo, Jaipur, 302001, India
10 University Centre for Research and Development (UCRD), Chandigarh University, Chandigarh, 140413, India

* Corresponding Author: Brij B. Gupta. Email:

Computers, Materials & Continua 2025, 84(3), 4755-4772. https://doi.org/10.32604/cmc.2025.064053

Received 03 February 2025; Accepted 14 May 2025; Issue published 30 July 2025

Abstract

Detecting cyber attacks in networks connected to the Internet of Things (IoT) is of utmost importance because of the growing vulnerabilities in the smart environment. Conventional models, such as Naive Bayes and support vector machine (SVM), as well as ensemble methods, such as Gradient Boosting and eXtreme gradient boosting (XGBoost), are often plagued by high computational costs, which makes it challenging for them to perform real-time detection. In this regard, we suggested an attack detection approach that integrates Visual Geometry Group 16 (VGG16), Artificial Rabbits Optimizer (ARO), and Random Forest Model to increase detection accuracy and operational efficiency in Internet of Things (IoT) networks. In the suggested model, the extraction of features from malware pictures was accomplished with the help of VGG16. The prediction process is carried out by the random forest model using the extracted features from the VGG16. Additionally, ARO is used to improve the hyper-parameters of the random forest model of the random forest. With an accuracy of 96.36%, the suggested model outperforms the standard models in terms of accuracy, F1-score, precision, and recall. The comparative research highlights our strategy’s success, which improves performance while maintaining a lower computational cost. This method is ideal for real-time applications, but it is effective.

---

Keywords

---