Open Access

ARTICLE

Safety-Aware Reinforcement Learning for Self-Healing Intrusion Detection in 5G-Enabled IoT Networks

Wajdan Al Malwi¹, Fatima Asiri¹, Nazik Alturki², Noha Alnazzawi³, Dimitrios Kasimatis⁴, Nikolaos Pitropakis^5,*

1 College of Computer Science, Informatics and Computer Systems Department, King Khalid University, Abha, Saudi Arabia
2 Department of Information Systems, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh, Saudi Arabia
3 Computer Science and Engineering Department, Yanbu Industrial College, Royal Commission for Jubail and Yanbu, Yanbu, Saudi Arabia
4 Blockpass ID Lab, Edinburgh Napier University, Edinburgh, UK
5 Department of Information Technology, Cybersecurity and Computer Science, The American College of Greece, Athens, Greece

* Corresponding Author: Nikolaos Pitropakis. Email:

Computers, Materials & Continua 2026, 87(2), 88 https://doi.org/10.32604/cmc.2026.074274

Received 07 October 2025; Accepted 20 January 2026; Issue published 12 March 2026

Abstract

The expansion of 5G-enabled Internet of Things (IoT) networks, while enabling transformative applications, significantly increases the attack surface and necessitates security solutions that extend beyond traditional intrusion detection. Existing intrusion detection systems (IDSs) mainly operate in an open-loop manner, excelling at classification but lacking the ability for autonomous, safety-aware remediation. This gap is particularly critical in 5G environments, where manual intervention is too slow and naive automation can lead to severe service disruptions. To address this issue, we propose a novel Self-Healing Intrusion Detection System (SH-IDS) framework that develops a closed-loop cyber defense mechanism. The main technical contribution is the integration of a deep neural network-based threat detector, which offers uncertainty-quantified predictions, with a safety-aware reinforcement learning (RL) engine formulated as a Constrained Markov Decision Process (CMDP). The CMDP explicitly models operational safety as cost constraints, and a new runtime safety shield actively adjusts any unsafe action proposed by the RL agent to the nearest safe alternative, ensuring operational integrity. Additionally, we introduce a composite utility function for the comprehensive evaluation of the system. Empirical analysis on the 5G-NIDD dataset demonstrates the superior performance of our framework: the detector achieves 98.26% accuracy, while the safe RL agent learns effective mitigation policies. Importantly, the safety shield blocked up to 70 unsafe actions under strict constraints, and analysis of the learned Q-tables confirms that the agent internalizes safety, avoiding overly disruptive actions, such as isolating nodes for minor threats. The system also maintains high efficiency with a compact model size of 121.7 KB and sub-millisecond latency, confirming its practical deployability for real-time 5G-IoT security.

---

Keywords

---