Open Access

ARTICLE

Context-Aware Spam Detection Using BERT Embeddings with Multi-Window CNNs

Sajid Ali¹, Qazi Mazhar Ul Haq^1,2,*, Ala Saleh Alluhaidan^3,*, Muhammad Shahid Anwar⁴, Sadique Ahmad⁵, Leila Jamel³

1 Department of Computer Science and Engineering, Yuan Ze University, Zhongli, Taiwan
2 Department of International Bachelor Program in Informatics, Yuan Ze University, Zhongli, Taiwan
3 Department of Information Systems, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh, Saudi Arabia
4 IRC for Finance and Digital Economy, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia
5 EIAS Data Science & Blockchain Laboratory, College of Computer and Information Science, Prince Sultan University, Riyadh, Saudi Arabia

* Corresponding Authors: Qazi Mazhar Ul Haq. Email: ; Ala Saleh Alluhaidan. Email:

(This article belongs to the Special Issue: Machine Learning and Deep Learning-Based Pattern Recognition)

Computer Modeling in Engineering & Sciences 2026, 146(1), 43 https://doi.org/10.32604/cmes.2026.074395

Received 10 October 2025; Accepted 23 December 2025; Issue published 29 January 2026

Abstract

Spam emails remain one of the most persistent threats to digital communication, necessitating effective detection solutions that safeguard both individuals and organisations. We propose a spam email classification framework that uses Bidirectional Encoder Representations from Transformers (BERT) for contextual feature extraction and a multiple-window Convolutional Neural Network (CNN) for classification. To identify semantic nuances in email content, BERT embeddings are used, and CNN filters extract discriminative n-gram patterns at various levels of detail, enabling accurate spam identification. The proposed model outperformed Word2Vec-based baselines on a sample of 5728 labelled emails, achieving an accuracy of 98.69%, AUC of 0.9981, F1 Score of 0.9724, and MCC of 0.9639. With a medium kernel size of (6, 9) and compact multi-window CNN architectures, it improves performance. Cross-validation illustrates stability and generalization across folds. By balancing high recall with minimal false positives, our method provides a reliable and scalable solution for current spam detection in advanced deep learning. By combining contextual embedding and a neural architecture, this study develops a security analysis method.

---

Keywords

---