iconOpen Access



An Enhanced Group Key-Based Security Protocol to Protect 5G SON Against FBS

Hoonyong Park1, TaeGuen Kim1, Daniel Gerbi Duguma1, Jiyoon Kim2, Ilsun You2,*, Willy Susilo3

1 Department of Information Security Engineering, Soonchunhyang University, Asan-si, 31538, Korea
2 Department of Financial Information Security, Kookmin University, Seoul-si, 02707, Korea
3 Institute of Cybersecurity and Cryptology, School of Computing and Information Technology, University of Wollongong, Wollongong, 2522, Australia

* Corresponding Author: Ilsun You. Email: email

Computer Systems Science and Engineering 2023, 45(2), 1145-1165. https://doi.org/10.32604/csse.2023.032044


Network operators are attempting many innovations and changes in 5G using self-organizing networks (SON). The SON operates on the measurement reports (MR), which are obtained from user equipment (UE) and secured against malware and userspace programs. However, the synchronization signal block that the UE relies on to measure the wireless environment configured by a base station is not authenticated. As a result, the UE will likely gauge the wrong wireless environment configured by a false base station (FBS) and transmit the corresponding MR to the serving base station, which poisons the data used for 5G SONs. Therefore, the serving base stations must verify the authenticity of the MR. The 3GPP has advocated numerous solutions for this issue, including the use of public key certificates, identity-based keys, and group keys. Although the solution leveraging group keys have better efficiency and practicality than the other two, they are vulnerable to security threats caused by key leaks via insiders or malicious UE. In this paper, we analyze these security issues and propose an improved group key protocol that uses a new network function, called a broadcast message authentication network function (BMANF), which validates broadcasted messages on behalf of the UE. The protocol operates in two phases: initial and verification. During the initial phase, the 5G core network distributes a shared secret key to the BMANF and UE, allowing the latter to request an authentication ticket from the former. During the verification phase, the UE requests the BMANF to validate the broadcasted messages received from base stations using the ticket and its corresponding shared key. For evaluation, we formally verified the proposed protocol, which was then compared with alternative methods in terms of computing cost. As a result, the proposed protocol fulfills the security requirements and shows a lower overhead than the alternatives.


1  Introduction

The introduction of enhanced mobile broadband and superior reliability (and low latency) with coverage for a high density of devices in the fifth-generation mobile networks has transformed several sectors and unlocked new services. The self-organizing network (SON) plays an important role in such networks by improving the overall network performance through self-x functions such as automatic configuration, healing, and optimization [1]. The primary goal of the SON is to supply intelligence to mobile networks to facilitate network organization, setup, optimization, and recovery. Furthermore, it minimizes total network complexities, capital expenditure, and operating expenses [25]. SONs anticipate that future networks will be paired with sophisticated algorithms to ensure higher network performance than that of previous generations of mobile networks. Such critical improvements combined with autonomous learning will enable future mobile networks to be much more proactive and adaptable than present mobile networks.

SON applications primarily target access network components, with important applications such as load balancing, mobility management, handover optimization, and backhaul optimization [6]. As a result, SON features operate at a high level by collecting and analyzing the measurement reports (MR) from user equipment (UE) connected to the network. However, there is a security risk of data poisoning, also known as SON poisoning, in which an attacker infiltrates the MR using a malicious UE or a false base station (FBS) [7]. This security issue can cause cell outages and signaling floods, as well as denial-of-service (DoS) attacks on both the UE and network [8]. UE is typically unable to verify the veracity of System Information (SI) messages transmitted from gNodeB or 5G base station (gNB) and may use them to communicate MR (possibly containing bogus information) to the gNB to which they are currently connected. Such unauthenticated messages would then lead to SON poisoning attacks.

To defend against such attacks, two major techniques, i.e., cryptographic and intrusion detection and prevention, can be considered. Our discussion is based on the previous line of defense, which was offered by 3GPP in three categories: public key certificate (PKC)-, identity-based cryptography (IBC)-, and group key-based schemes [7]. The first two schemes use asymmetric keys to protect over-the-air intruders that replay previously recorded SI messages or send malevolently constructed ones aiming to poison the SONs. Although these approaches play a significant role in authenticating broadcast message (BM) without presharing secret keys, they possess a high computational overhead because of their expensive operations. Additionally, while the primary solution requires costly public key infrastructure (PKI) to manage digital signatures and public key encryption, the IBC lacks practicality and has a key escrow problem [9]. In contrast, the group key-based scheme depends on the collaboration of the serving network (SN) and home network (HN) for the dynamic supply of session keys and computation of message authentication codes between the UE and gNB. Nevertheless, the solution has security concerns regarding insider threats. An attacker with a genuine UE can gain access to a shared group key from the different groups to which it belongs. We explain these security mechanisms in depth in Section 2.

Inspired by the previous technique presented, we analyze the security problem and propose an improved version of it. In each group, the enhanced protocol uses a new network function to verify the BM on behalf of the UE. The following are the main contributions of this paper:

■   By analyzing and improving the security concerns of the group key-based solution, we design a secure and authentication ticket-based protocol that prohibits the UE from sharing a group key. Moreover, through the Burrows–Abadi–Needham (BAN) logic [10] and Automated Validation of Internet Security Protocols and Applications (AVISPA) [11], we formally verify the protocol and show that it satisfies the security requirements of mutual authentication and secure key exchange.

■   In terms of performance and security, we compare the proposed protocol to group key-, PKC-, and IBC-based solutions. As a result, we demonstrate that the proposed scheme achieves a better performance than the alternative solutions.

The rest of the paper is structured as follows: Section 2 discusses the 5G SON security threats and their proposed countermeasures. Section 3 describes our proposed protocol, and Section 4 presents its formal verification. Section 5 outlines a comparative analysis of the proposed protocol compared with other proposed solutions. Finally, Section 6 concludes the paper.

2  Security Threats and Existing Countermeasures of SON in 5G

This section discusses two security vulnerabilities in the context of 5G: authentication relay attack and SON data contamination. Furthermore, we describe the key cryptographic schemes to counter these attacks (e.g., PKC-, IBC-, and group key-based methods).

2.1 Security Threats

Wireless attackers can send maliciously crafted MR through the UE using their software-defined radio (SDR) or build a wireless environment to generate distorted MR around genuine UE through FBS, as illustrated in Fig. 1a. In more detail on the latter, the FBS may collect cell IDs of adjacent regular base stations and use one of them to impersonate a real base station. Subsequently, it can create a radio environment disguised as a genuine base station to trick UE into generating an MR based on that environment and transmitting it to the serving base station. This attack is possible because the UE cannot verify the received SI and thus cannot determine whether the adjacent base station is false.


Figure 1: An illustration of (a) SON contamination and (b) Authentication relay attacks

However, there is a more complex scenario in which the FBS and malicious UE are far apart and connected by local or wide area networks to form vicious links. First, the FBS induces access to the victim UE and transmits an initial registration request message to the malicious UE. The malicious UE then transmits the received message to the core network through a remote legitimate base station. Then, in reverse order, the FBS and malicious UE forward the response message sent by the core network to the UE to complete the authentication. By relaying the base station physically separated in this way, an attacker may cause the UE to report the signal strength of the FBS to the legitimate base station, inducing confusion in the network SON configuration. Fig. 1b shows the authentication relay attack.

2.2 Existing Countermeasures

Few studies (such as [12,13]) proposed various mechanisms to counteract the FBS. However, in this paper, we focus on those proposed by 3GPP: PKC-, IBC-, and group key-based schemes [7].

2.2.1 Public Key Certificate-based Solutions

PKC-based countermeasures enable the UE to validate the authenticity of the BM that gNBs disseminate without the need for sharing the signing key. During the “Registration Accept” and “Location Update Accept” messages exchanged between AMF and UE, the former sends the latter a list of tracking area IDs along with the corresponding public keys and lifespan. The network operator also provides the matching private keys to the NR, which it will use to sign the SI with other information like time counter, downlink frequency, and physical cell ID. The inclusion of this extra information hinders replay attacks. The NR then broadcasts the digital signature along with the least significant bits of time counter (to address errors introduced because of time counter differences between the UE and access network) and the SI. The UE verifies the digital signature and time counter using the stored public key for that particular tracking area. Public key-based solutions can also use the PKI to protect the SI. The gNB signs the BM with its private key and sends it to the UE along with a plain message and its certificate. The UE then verifies the received certificate with the root certificate provided at the manufacturing time or during the “Registration Accept” message. In both of these solutions, digitally signing SI can be delegated to a new network function called digital signing network function (DSnF).

2.2.2 Identity-based Cryptographic-based Solutions

IBC-based countermeasure is another method for SI verification. In this technique, the core network provides different credentials to the NR and UE for the certificateless signature algorithm. It also provides the NR with a private signing key for signing SI and a public validation token unique to each cell for validating the signature to the UE. An additional solution in this category leverages the private key generator (PKC) along with the public and private keys. The public key is programmed into the UE, either at manufacturing time or through the non-access stratum security mode command complete message, whereas the private key is kept secret. Subsequently, the PKC use the identity of the gNB and its private key to derive a signing key for the gNB. Once the BM with a digital signature is received, the UE verifies it using PKC’s public key and gNB’s identity (which can be reconstructed from the BM).

2.2.3 Group Key-based Solution

The group key-based method for protecting the BM operates among the UE, SN, and HN of the 5G environment. The SN automatically supplies the UE (specifically mobile equipment (ME)) with keys of the gNBs, which are ciphered with the assistance of the HN and cannot be deciphered by the ME. As shown in Fig. 2, the solution consists of three phases: protection key agreement and transfer (PKAT), protection area (PA) information provisioning (PAIP), and cell authentication procedure. In the first phase, the protection key CKp is agreed upon between the HN (AUSF/UDM) and UE, and the same key is transferred to the SN (AMF/SEAF) for the subsequent procedures. It is worth noting that because the long-term key is found only in the universal subscriber identity module (USIM) and HN, only the USIM can correctly compute CKp. During the PAIP procedure, the SN provides the encrypted root keys of groups (called share root key group (SRKG)) in the PA to the UE, which optimizes the number of encrypted keys supplied to the ME. The SRKG is represented by a group key identifier (GKI), whereas the gNBs in an SRKG are identified by a unique group node identifier (GNI). The SN provides the GKI and GNI to the corresponding gNB and computes KBS = HMAC-SHA-256 (KRBS, <GKI, GNI>), where KRBS is a root key shared by a group of gNBs. The SN then provides KBS to the corresponding gNB. Once these initial setups have been completed, the SN obtains the list of PA information (including GKI, EKRBS, and lifetime, where EKRBS is KRBS encrypted with CKp) based on the UE’s request for registration. It then returns the registration response with the PA information back to the UE. In the final phase, the gNB computes the message authentication code (MACi) of the SI (including GKI and GNI) and downlink frequency with KBS. It then broadcasts the GKI and GNI with the MACi, where the UE receives and checks whether the GKI exists in the PA information. If so, the ME sends the EKRBS, <GKI, GNI> to the USIM, in which the latter decrypts the EKRBS (using CKp) to KRBS, computes the KBS based on KRBS, and computes the expected message authentication code (xMACi). Subsequently, the USIM sends the xMACi to the ME to compare it with the MACi.


Figure 2: Group key-based solution to counteract the FBS

3  Proposed Protocol

The proposed protocol is an improved version of the group key-based solution proposed in [7]. Although the original group key-based technique has some advantages, including enhanced computing efficiency and lessened group administration costs, it has a substantial security flaw, primarily because the root key KRBS (technically the EKRBS) is shared with the UE. As a result, if an attacker compromises one group member or joins multiple groups with a legitimate UE (but with malicious intent), the group key can be revealed, allowing legitimate gNBs to be masqueraded. Hence, we propose a network function called a BM authentication network function (BMANF) to tackle this problem and validate the BM instead of the UE. In addition to this, we addressed other security concerns with the solution. Tab. 1 outlines the descriptions of the notations used in this protocol.


3.1 System Model

The main difference between the proposed protocol and the 3GPP candidate solution is validating the BM. In the latter case, the UE performs the verification process, whereas the former uses a BMANF in each group to verify the BM. Fig. 3 shows the system model of the proposed protocol.


Figure 3: System model for the proposed protocol

Before transmitting the MR to the serving gNB, the UE contacts the BMANF to verify the received BM using its HMAC. The BMANF uses its own KRBS to derive the KBS of each base station, confirms the HMAC, and informs the UE about the result. As a result, even if the FBS delivers a BM, the BMANF may validate it before the UE transmits the MR to the gNB, preventing the gNB from receiving an erroneous MR. Hence, handover attempts to the FBS are blocked, and the quality of data for the SON functions is preserved.

3.2 Threat Model

To securely exchange messages over insecure channels, protocols must be safe from passive attackers who compromise confidentiality by eavesdropping on messages and active attackers who violate the integrity and availability of communications by modifying and deleting messages. The attackers can also be malicious insiders or unauthorized outsiders. The Dolev–Yao (DY) model [14] is best suited to modeling the threats emanating from such security problems. A DY attacker is a powerful adversary that can eavesdrop on all messages transmitted, including those between the UE and gNB, and knows all the proposed protocol procedures. Furthermore, it can exchange messages with communication participants and forge, retransmit, or even delete messages. However, this attacker cannot decrypt a ciphered message without the correct encryption key, recover the result of a one-way hash function, and guess random numbers.

3.3 Security Requirements

The primary goal of the proposed protocol is to protect the BM authentication procedure in the 5G environment. Therefore, it must meet essential security requirements like a secure key exchange and mutual authentication. Secure key exchange prescribes a safe key establishment procedure that will be used to encrypt and guard communications, whereas mutual authentication refers to the property that entities establish each other’s identity before real communication begins.

3.4 Assumptions

Several gNBs within the tracking area managed by one AMF constitute different groups. The root key, KRBS, is shared by all the gNBs in each group. The KRBSis provisioned to the gNB and updated periodically by the AMF. Each gNB derives the KBS from the KRBS using their group identifier and calculating the HMAC of the message when transmitting a BM. The formula to derive the KBS is shown in Eq. (1).


The UE and AMF know the algorithm to derive the shared key between the BMANF and UE, KBMANF-UE, from KAMF. The BMANF communicates with the AMF by establishing a secure channel. Furthermore, it is assumed that all nodes are time synchronized.

3.5 The Initial Phase

Fig. 4 shows the procedure of the initial phase of the proposed protocol. In this phase, the KBMANF-UE is distributed using the existing 5G registration mechanism. Following the AS security activation, the UE requests a ticket from the BMANF, which it uses for authentication in subsequent sessions. The following are the specifics of the initial phase.


Figure 4: Initial phase of the proposed protocol

Step 1: This step follows the registration request procedure defined in 3GPP TS 38.331 [15], TS 38.413 [16], and TS 33.501 [17].

Steps 2a–2c: The AMF derives the KBMANF-UE from the KAMF and transmits message 2b to the UE following the registration accept procedure defined in the standard, such as in Step 1, and then delivers the KBMANF-UE to the BMANF.

Steps 3a and 3b: The UE derives the KBMANF-UE from the KAMF in the same way that the AMF did. Then, using the KBMANF-UE, it encrypts the Temporary Mobile Subscriber Identity (TMSI) by including IDUE, IDAMF, and Timestamp1. It should be noted that IDUE is a UE’s pseudo identity, a structure comparable to the 5G globally unique temporary identification (5G-GUTI). The actual computation of IDUE is outside the scope of this study. Finally, the UE requests a ticket by sending the ticket request message that includes the IDUE and encrypted TMSI to the BMANF.

Steps 4a and 4b: The BMANF decrypts the message with the KBMANF-UE, verifies Timestamp1, and if valid, creates KTicket and TicketUE. The TicketUE is constructed as shown in Eq. (2).


Next, the BMANF prepares a “Ticket Response” message by encrypting the IDUE, IDBMANF, KTicket, TicketUE, ExpireT, and Timestamp2 with the KBMANF-UE and transmits it to the UE.

Step 5: The UE verifies the validity of Timestamp2, and if the verification succeeds, it stores the KTicket and TicketUE. Following that, the UE and BMANF delete KBMANF-UE from their memory to assure perfect forward secrecy.

3.6 The Verification Phase

The verification phase begins once the initial phase is completed. When the UE triggers the MR transmission condition set via MeasConfig, before transmitting the MR to the serving gNB, the UE dispatches the BM of the adjacent gNBs used to create the MR and its HMAC to the BMANF for a verification request. According to the verification result, the UE transmits an MR to the serving gNB or creates a new one. When the UE requests verification from the BMANF, it transmits the TicketUE together for authentication. Fig. 5 shows the procedure of the verification phase, and its description is presented as follows:


Figure 5: Verification phase of the proposed protocol

Steps 0a and 0b: The gNBs use their KBS to generate the HMAC of the BM. The gNBs transmit the BM with the corresponding HMAC values to nearby UE.

Steps 1a and 1b: The UE creates a list of BMs and HMAC values received from different gNBs that will be used to generate the MR. To request the BMANF for BM verification, the UE encrypts the BM, its HMAC list, and Timestamp3 with KTicket and transmits the TicketUE together.

Steps 2a and 2b: The BMANF decrypts the TicketUE using its secret key, KBMANF, and then reads the KTicket. Next, it verifies Timestamp3 and, if valid, confirms the HMAC of the BMs. If confirmation succeeds, the BMANF transmits a success message with Timestamp4. If not, the BMANF creates a list of filed BMs and sends it along with a message of failure.

Step 3: The UE verifiesTimestamp4 after receiving the message verification response message. If valid, it transmits an MR to the serving base station. If not, the UE generates an MR, except for the gNB that failed verification, and transmits it to the serving gNB.

Tab. 2 summarizes the approaches, pros, and cons of the solutions proposed by 3GPP and our proposed protocol.


4  Formal Security Verification

This section analyzes the security of the proposed protocol using two well-known verification tools: BAN logic and AVISPA.

4.1 BAN Logic-Based Formal Verification

BAN logic, named after its three creators, Burrows, Abadi, and Needham, uses distinct notations and rules to verify security protocols based primarily on belief deductions. BAN logic’s verification procedure consists of four steps: idealization (modeling the protocol using formal logic), assumption (setting initial realistic security assumptions), goals (defining the security objectives that the protocol is expected to meet), and derivation (deriving the security goals from the rules, assumptions, and intermediate results of the derivation). The BAN logic notations and rules can be found in [18] and [19]. The reader is also referred to Appendices B and C for the notations and rules, respectively. The following is the formal verification of the proposed protocol using the BAN logic.

4.1.1 Initial Phase

Idealization. Here, all messages that are not protected (for instance through encryption) are excluded. The idealized initial phase of the proposed protocol is shown as follows.



Assumption. The assumptions that the BMANF and UE make regarding security keys and the freshness of the timestamps are shown in (IA1) and (IA2), and (IA3)(IA5), respectively.


BMANF |#(TS1) (IA2)


UE |#(TS2) (IA4)


Goal. The security goals of the initial phase of the protocol regarding the secure exchange of identities and the preshared secret key (KBMANF-UE) are formulated in (IG1)(IG4), respectively. (IG5) denotes a successful ticket key allocation from the BMANF to the UE.






Derivation. As shown in (ID1)(ID12), the security goals are derived by applying the idealization, assumptions, and BAN logic rules.

From (II1):






From (II2):







UE | UEKTBMANF by (D11),(A5),JR (ID12)

From the aforementioned derivations (ID1)(ID12), it is possible to see that all goals have been derived. The following theorem and lemmas illustrate this fact.

Theorem 1: The initial phase of the proposed protocol is secure.

Proof: From Lemma 1–1 and Lemma 1–2, the defined goals are satisfied, and hence, the initial phase of the proposed protocol is secure.

Lemma 1–1: The initial phase of the proposed protocol supports mutual authentication.

Proof: The derived belief (ID4) shows that the BMANF authenticates the UE, and (ID9) shows that the UE authenticates the BMANF. From this, the initial phase of the proposed protocol can provide mutual authentication.

Lemma 1–2: The initial phase of the proposed protocol provides secure key exchange.

Proof: The BMANF trusts the KBMANF-UE assigned by the AMF as defined in (IA1). Furthermore, the UE believes that the KBMANF-UE can be derived from KAMF as defined in (IA3). Hence, (ID5) and (ID10) show that both UE and BMANF can believe KBMANF-UE indirectly.

4.1.2 Verification Phase

Idealization. The idealization for the verification phase is given as follows.




Assumption. The first five assumptions are for the BMANF, whereas the remaining two assumptions are for the UE concerning symmetric keys and the freshness of the ticket and timestamp.




BMANF |#(TS1) (AA4)



UE |#(TS2) (AA7)

Goal. Goals (AG1)(AG4) are for the BMANF, whereas (AG5) and (AG7) are for the UE.








Derivation. The aforementioned goals (AG1)(AG7) are derived as follows.

From (AI1):






From (AI2):




BMANF | UE | IDUE by (D8),BC (AD9)


BMANF | UE | [BM,BMANFKBSgNB] by (D6),(A5),MM,BC (AD11)

BMANF | UE | BM by (D11),BC (AD12)

From (AI3):







According to the aforementioned derivation (AD1)(AD18), all goals (AG1)(AG7) are achieved. The following theorem and lemmas illustrate these goals.

Theorem 2: The verification phase of the proposed protocol is secure.

Proof: From Lemma 2–1 to Lemma 2–3, the defined goals are satisfied, and hence, the verification phase of the proposed protocol is secure.

Lemma 2–1: The verification phase of the proposed protocol provisions mutual authentication.

Proof: The derived belief (AD9) shows that the BMANF authenticates the UE, and (AD16) shows that the UE authenticates the BMANF. From this, it is possible to show that the protocol provides mutual authentication.

Lemma 2–2: The secret key KTicket is successfully exchanged between the UE and BMANF.

Proof: The UE and BMANF have a direct belief in the secret key KTicket through (AA6) and (AD5). Conversely, the indirect belief of the UE and BMANF in the secret key KTicket can be proved with (AD10) and (AD18).

Lemma 2–3: The verification phase of the proposed protocol provides confidentiality and integrity.

Proof: From the aforementioned Lemma 2–2, the secret key KTicket is successfully communicated between the UE and BMANF. Hence, the verification phase of the proposed protocol provides confidentiality and integrity with KTicket.

In conclusion, both initialization and verification phases of the proposed protocol are proven to be secure through Theorems 1 and 2.

4.2 AVISPA-based Simulation

The BAN logic has limitations when it comes to accurately specifying a protocol, particularly during the idealization step [20]. Accordingly, automated formal verification tools alongside BAN logic are often used to complement the limitations. In this section, we use AVISPA to verify the proposed protocol formally. AVISPA uses a unique language called the high-level specification language (HLPSL) to describe the protocol and security property. The HLPSL specification is translated into the intermediate format (IF) through the HLPSL2IF. The translated IF specification is entered into each verification module of the AVISPA to analyze the defined security goals.

The HLPSL specification consists of three roles: basic, session, and environment. The basic role defines protocol participants’ specifications and initial information and expresses message exchange between protocol participants. Session role defines parameters and channels for protocol session participants. The environment role declares the protocol and overall specification of sessions and attackers. The basic roles of each agent are depicted in Figs. 6 and 7, and the AVISPA verification result for the proposed protocol using the on-the-fly model checker and constraint-logic-based attack searcher is shown in Fig. 8. The results confirm that the proposed protocol is secure for the set of security properties.


Figure 6: Basic roles for (a) UE and (b) BMANF


Figure 7: Basic roles for gNB


Figure 8: AVISPA verification result for the proposed protocol

5  Comparative Analysis

This section presents the performance and security comparison results of the proposed protocol against the PKC-, IBC-, group key-based, and DSnF schemes [12]. Tab. 3 shows the latency of the cryptographic operations, and Tab. 4 compares the performance and security of these solutions against our protocol. We measure the computational overhead using the Apple MacBook Pro M1 max 64 GB RAM.



On the basis of the results of Tabs. 3 and 4, Fig. 9 depicts a series of graphs that illustrate the computational overhead against the density of the gNBs (with f ranging from low to medium to high), broadcast per second (with n ranging from low to medium to high), and round-trip time (with both f and n ranging from low to medium to high). The greater the number of adjacent base stations and the shorter the broadcast period, the more cryptographic operations must be performed. Furthermore, the higher the roundtrip time, the higher the latency of the proposed protocol. To compare the overhead of each method with the environment, an experiment was conducted. It is worth noting that the overhead in the graph is the total computational cost of the UE, gNBs, and network functions (BMANF and DSnF).


Figure 9: Performance comparison of different FBS defense mechanisms

According to Fig. 9, the performance of the PKC, IBC, and DSnF schemes persistently shows significantly higher overhead as the density of the gNBs and their broadcast frequency increase. Such high loads are exhibited mainly because of the costly certificate transport and heavy Elliptic Curve Digital Signature Algorithm for the PKC and Elliptic Curve-Based Certificateless Signatures for Identity-Based Encryption for the IBC. Compared with the group key-based solutions (the original and enhanced), the disparity in computational cost widens as the density and broadcast frequency increases, which makes the PKC, IBC, and DSnF solutions less desirable.

Concerning the group key-based schemes, the original group key-based solution shows a slightly lower overhead as the density of the gNBs and broadcast frequency grow. In terms of the round-trip time, the proposed solution exhibits a higher overhead (compared with the group key-based solution) due to the addition of the BMANF, which is located near the gNB control unit). However, the burden of the UE (which is computationally resource constrained ) in the proposed solution is less. In contrast, the UE in the other solutions perform frequent computations as the BM transmission period is short. Furthermore, the proposed protocol addresses critical security flaws found in the group key-based scheme while also satisfying all the four security requirements itemized in Tab. 4. Therefore, the proposed protocol is secure, efficient, and suitable, especially in environments where the computing resource of the UE is limited.

6  Conclusions

This paper proposes a security protocol that can prevent the FBS and improve the quality of data for the SON functions, thereby safeguarding networks from SON-poisoning attacks. The suggested protocol is divided into two phases: initial and verification. During the first phase, the AMF distributes a shared key KBMANF-UE to the UE and BMANF. Following that, the UE securely requests an authentication ticket from the BMANF using the shared key. The UE then requests that the BMANF validate the BM received from the gNBs using the authentication token obtained during the verification phase. The UE determines whether or not to transmit the MR to the base station based on the verification result. The proposed protocol has also been formally verified using the BAN logic and AVISPA. The verification findings reveal that initial and verification phases of the protocol are secure and satisfy the specified security requirements. Furthermore, in terms of computational cost, we compared our protocol with the PKC-, IBC-, and group key-based protocols given in 3GPP TR 33.809. As a result, particularly in resource-limited contexts, our technique has been demonstrated to be more efficient and beneficial than competing solutions detailed in Section 2.2. However, the proposed approach has some limitations that cannot be avoided when a hostile UE using SDR sends a counterfeit MR to the serving gNB. Other security solutions, such as an intrusion detection system, are required for such special cases because it is difficult to respond to an FBS using only cryptographic techniques. As follow-up research, we would like to use the BMANF to investigate machine learning-based techniques (such as designing anomaly detection algorithms) for detecting the MR emitted by hostile UE.

Funding Statement: This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00952, Development of 5G Edge Security Technology for Ensuring 5G+ Service Stability and Availability, 100%)

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.


 1.  O. G. Aliu, A. Imran, M. A. Imran and B. Evans, “A survey of self-organization in future cellular networks,” IEEE Communications Surveys Tutorials, vol. 15, no. 1, pp. 336–361, 2013. [Google Scholar]

 2.  P. V. Klaine, M. A. Imran, O. Onireti and R. D. Souza, “A survey of machine learning techniques applied to self-organizing cellular networks,” IEEE Communications Surveys Tutorials, vol. 19, no. 4, pp. 2392–2431, 2017. [Google Scholar]

 3.  P. V. Klaine, O. Onireti, R. D. Souza and M. A. Imran, “The role and applications of machine learning in future self-organizing cellular networks,” in Next-Generation Wireless Networks Meet Advanced Machine Learning Applications, Hershey, Pennsylvania, USA: IGI Global, pp. 1–23, 2019. https://www.igi-global.com/about/. [Google Scholar]

 4.  M. D. Cia, F. Mason, D. Peron, F. Chiariotti, M. Polese et al., “Using Smart City Data in 5G Self-Organizing Networks,” IEEE Internet of Things Journal, vol. 5, no. 2, pp. 645–654, 2018. [Google Scholar]

 5.  A. Imran, A. Zoha and A. Abu-Dayya, “Challenges in 5G: How to empower SON with big data for enabling 5G,” IEEE Network, vol. 28, no. 6, pp. 27–33, 2014. [Google Scholar]

 6.  H. Fourati, R. Maaloul, L. Chaari and M. Jmaiel, “Comprehensive survey on self-organizing cellular network approaches applied to 5G networks,” Computer Networks, vol. 199, pp. 108435, 2021. [Google Scholar]

 7.  SA3, “3rd generation partnership project; technical specification group services and system aspects study on 5G security enhancement against false base stations (FBS) (Release 17),” 3GPP, 2022. [Online]. Available: https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3539. [Google Scholar]

 8.  A. Shaik, R. Borgaonkar, S. Park and J. P. Seifert, “On the impact of rogue base stations in 4G/LTE self organizing networks,” in Proc. WiSec, New York, NY, USA, pp. 75–86, 2018. [Google Scholar]

 9.  K. Emura, S. Katsumata and Y. Watanabe, “Identity-based encryption with security against the KGC: A formal model and its instantiations,” Theoretical Computer Science, vol. 900, pp. 97–119, 2022. [Google Scholar]

10. M. Burrows, M. Abadi and R. M. Needham, “A logic of authentication,’’ Proceeding of the Royal Society of London: A Mathematical and Physical Sciences, vol. 426, no. 1871, pp. 233–271, 1989. [Google Scholar]

11. A. Armando, D. Basin, Y. Boichut, Y. Chevalier, L. Compagna et al., “AVISPA tool for the automated validation of internet security protocols and applications,” in Proc. CAV, Edinburgh, Scotland, UK, pp. 281–285, 2005. [Google Scholar]

12. H. Gao, Y. Zhang, T. Wan, J. Zhang and H. Duan, “On evaluating delegated digital signing of broadcasting messages in 5G,” in Proc. GLOBECOM, Waikoloa, HI, USA, pp. 1–7, 2019. [Google Scholar]

13. P. K. Nakarmi, M. A. Ersoy, E. U. Soykan and K. Norrman, “Murat: Multi-RAT false base station detector,” arXiv e-prints, vol. 2102, no. 8780, pp. 1–13, 2021. [Google Scholar]

14. D. Dolev and A. Yao, “On the security of public key protocols,” IEEE Transaction on Information Theory, vol. 29, no. 2, pp. 198–208, 1983. [Google Scholar]

15. RAN2, “3rd generation partnership project; technical specification group radio access network; NR; radio resource control (RRC) protocol specification (Release 16),” 3GPP, 2021. [Online]. Available: https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3197. [Google Scholar]

16. RAN3, “3rd generation partnership project; technical specification group radio access network; NG-RAN; NG application protocol (NGAP) (Release 16),” 3GPP, 2021. [Online]. Available: https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3223. [Google Scholar]

17. SA3, “3rd generation partnership project; technical specification group services and system aspects; security architecture and procedures for 5G system (Release 17),” 3GPP, 2022. [Online]. Available: https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3169. [Google Scholar]

18. J. Kim, D. G. Duguma, P. V. Astilo, H. Y. Park, B. Kim et al., “A formally verified security scheme for inter-gNB-DU handover in 5G vehicle-to-everything,” IEEE Access, vol. 9, pp. 119100–119117, 2021. [Google Scholar]

19. D. G. Duguma, J. Kim, S. Lee, N. S. Jho, V. Sharma et al., “A lightweight D2D security protocol with request-forecasting for next-generation mobile networks,” Connection Science, vol. 34, no. 1, pp. 362–386, 2021. [Google Scholar]

20. C. Boyd and W. Mao, “On a limitation of BAN logic,” in Proc. EUROCRYPT, Lofthus, Norway, pp. 240–247, 1993. [Google Scholar]

Appendix A Abbreviations


Appendix B Notations of BAN Logic


Appendix C Rules of BAN Logic


Cite This Article

APA Style
Park, H., Kim, T., Duguma, D.G., Kim, J., You, I. et al. (2023). An enhanced group key-based security protocol to protect 5G SON against FBS. Computer Systems Science and Engineering, 45(2), 1145-1165. https://doi.org/10.32604/csse.2023.032044
Vancouver Style
Park H, Kim T, Duguma DG, Kim J, You I, Susilo W. An enhanced group key-based security protocol to protect 5G SON against FBS. Comput Syst Sci Eng. 2023;45(2):1145-1165 https://doi.org/10.32604/csse.2023.032044
IEEE Style
H. Park, T. Kim, D.G. Duguma, J. Kim, I. You, and W. Susilo "An Enhanced Group Key-Based Security Protocol to Protect 5G SON Against FBS," Comput. Syst. Sci. Eng., vol. 45, no. 2, pp. 1145-1165. 2023. https://doi.org/10.32604/csse.2023.032044

cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1102


  • 545


  • 0


Share Link