Open Access

REVIEW

AI-Native Security for 5G and 6G O-RAN: A Survey of Learning-Based Defense against False Base Stations, DDoS, and Signaling Storms

I Wayan Adi Juliawan Pawana^1,2, Vincent Abella², Eldridge Aaron Miole², Hoonyong Park³, Ilsun You^2,4,*

1 Department of Electrical Engineering, Udayana University, Badung, Indonesia
2 Department of Cyber Security, Kookmin University, Seoul, Republic of Korea
3 AUTOCRYPT Co., Ltd., Seoul, Republic of Korea
4 KMU Global Research Center for ICT Convergence Security, Kookmin University, Seoul, Republic of Korea

* Corresponding Author: Ilsun You. Email:

(This article belongs to the Special Issue: The Evolution of Cybersecurity and AI: Surveys and Tutorials)

Computer Modeling in Engineering & Sciences 2026, 148(3), 6 https://doi.org/10.32604/cmes.2026.087445

Received 16 June 2026; Accepted 28 August 2026; Issue published 28 September 2026

Abstract

Fifth- and sixth-generation cellular networks remain exposed to a spectrum of threats ranging from the stealthy and targeted false base station (FBS) to the systemic distributed denial-of-service (DDoS) and signaling-storm attacks. The elevation of artificial intelligence (AI) to a native network capability by the International Telecommunication Union Radiocommunication Sector (ITU-R) framework for International Mobile Telecommunications-2030 (IMT-2030) has made learning-based defense the dominant response. This survey examines that response across the three threats jointly, advancing as a design position the view that effective defense is best served when AI is integrated into the radio access network (RAN) and core as a native function rather than appended as an external classifier. We organize the literature by five technique families, namely classical machine learning, deep learning, generative models, federated learning, and large language and agentic models, and for each we record not only how it detects an attack but where in the Open RAN (O-RAN) architecture it is deployed, mapping the surveyed methods onto the user equipment, the near-real-time and non-real-time RAN Intelligent Controller, and the 5G-core network function. Three observations organize the account: the three threats form a data-scarcity gradient that illuminates the progression of techniques; the deployment dimension, the question of where and how, is the one the primary literature most often omits and the one a practitioner most needs; and several threat-technique combinations that the architecture permits remain unexplored. Drawing on a semi-systematic review of 177 studies, the survey consolidates the datasets and reported performance that make learning feasible, examines the adversarial robustness of the defenses themselves, and sets out a forward agenda toward quantum-AI-native defense within the RAN.

---

Keywords

---