Reliable Low-Latency Task Offloading and Resource Allocation Method for Space-Air-Ground Integrated Networks

1  Introduction

1.1 Research Background and Motivation

As the core network architecture for the 6G era, Space-Air-Ground Integrated Networks (SAGIN) are profoundly transforming the deployment methods and service capabilities of distributed urban sensing systems [1]. By integrating Low Earth Orbit (LEO) satellite constellations in the space layer, Unmanned Aerial Vehicle (UAV) swarms in the air layer, and terrestrial edge servers in the ground layer, SAGIN constructs a multi-layered, wide-coverage, and highly reliable computing infrastructure. This provides unprecedented support capabilities for applications such as environmental monitoring, traffic management, public safety, and emergency response in smart cities [2].

Recent studies have also moved toward graph-based DRL, multi-agent coordination, and constrained/safe RL for SAGIN and LEO edge computing [3–7]. These works reinforce the need for topology-aware and constraint-aware resource management; however, many of them emphasize representation, coordination, or performance-oriented offloading, whereas our focus is to make task-level reliability cost, redundancy, and low-latency inference jointly explicit under a unified offloading formulation.

However, the heterogeneity and dynamics of SAGIN also present unprecedented challenges for task offloading and resource allocation. Unlike the traditional “terminal-edge-cloud” three-tier architecture, each layer of SAGIN possesses unique computing capacity, communication characteristics, and reliability features: satellites offer wide-area coverage but suffer from high communication latency and weather-dependent link reliability; UAVs feature flexible deployment but are constrained by battery life and dynamic positional changes; terrestrial edge servers provide strong computing power but have a limited coverage range. How to make optimal task offloading decisions among these heterogeneous resources while ensuring end-to-end reliability has become a critical problem to be solved.

1.2 Challenges Faced by Existing Methods

Existing research on SAGIN task offloading and resource allocation can be broadly categorized into three paradigms, each exhibiting significant limitations when applied to space-air-ground integrated scenarios with strict reliability requirements.

Heuristic Methods: Such as nearest node selection and load-balancing round-robin, though computationally lightweight, these methods offer the advantage of computational simplicity. However, these methods are typically based on static rules and fail to adapt to the dynamic characteristics of SAGIN—orbital satellite motion, changing UAV positions, and fluctuating link conditions all require decision methods to possess real-time adaptive capabilities. For instance, simply selecting the nearest edge server might work well on the ground but could lead to task interruption when a satellite visibility window closes.

Optimization Methods: These methods model task offloading as mathematical programming problems, utilizing techniques like Lyapunov optimization [8], game theory [9], and convex optimization [10]. While these approaches can provide theoretical optimality guarantees under certain assumptions, accurately modeling SAGIN’s cross-layer decision space and dynamic constraints is extremely difficult. Satellite orbital constraints, UAV battery constraints, and cross-layer link reliability constraints are mutually coupled, forming high-dimensional non-convex optimization problems that traditional methods struggle to solve within a limited timeframe.

Learning Methods: Deep Reinforcement Learning (DRL) has gained widespread attention for its ability to learn optimal policies through environmental interaction [11]. However, several key limitations remain unresolved:

•   Cross-Layer Decision Complexity: The action space in SAGIN includes local execution, multiple terrestrial edge servers, multiple UAVs, and multiple satellites. The action space is highly dimensional, and the computation latency of each option varies significantly (ranging from millisecond-level edge execution to hundreds of milliseconds for satellite execution). Traditional DRL methods struggle to effectively learn such a complex decision space.

•   Heterogeneous Link Reliability: Communication links across different layers in SAGIN exhibit distinct reliability characteristics. Terrestrial links suffer from multi-path fading, air-to-ground links are affected by Line-of-Sight (LoS)/Non-Line-of-Sight (NLoS) switching, and satellite-to-ground links are impacted by rain attenuation. Existing methods often employ simplified reliability models that fail to accurately characterize these heterogeneous features.

•   Dynamic Resource Availability: Satellite visibility windows are limited, UAV batteries continuously deplete, and edge server loads change dynamically. These time-varying constraints make traditional static resource allocation methods difficult to apply.

•   Limited Theoretical Support: Convergence and feasibility guarantees for DRL methods addressing SAGIN cross-layer constrained optimization problems are often not established, leading to unstable performance during actual deployment.

1.3 Main Contributions

To address the aforementioned challenges, this paper proposes a novel framework named DRL-RA (Deep Reinforcement Learning and Reliability-Aware), which synergistically integrates a Dueling Double Deep Q-Network (D3QN) with a Reliability-Aware Multi-Objective Optimization Framework (RA-MOOF), specifically designed for task offloading and resource allocation in Space-Air-Ground Integrated Networks. The main contributions of this paper are summarized as follows:

1.   We establish a complete SAGIN task offloading system model, encompassing a cross-layer task offloading decision space, heterogeneous communication models (device-to-edge, device-to-UAV, device-to-satellite, and UAV-to-satellite links), hierarchical computation models, and cross-layer reliability metrics. This model accurately captures the heterogeneity and dynamics of SAGIN.

2.   We propose a cross-layer reliability measurement method tailored for SAGIN, comprehensively considering delay reliability, node availability (satellite visibility windows, UAV battery constraints, server failure models), and link reliability (differential modeling for terrestrial, air-to-ground, and satellite-to-ground links). We introduce a smooth reliability proxy to provide dense reward signals and improve the numerical stability of Lagrangian multiplier updates.

3.   We design a cross-layer decision module based on the Dueling Double Deep Q-Network. The action space covers four types of target options: local execution, terrestrial edge servers, UAVs, and satellites. The dueling network architecture and double Q-learning mechanism effectively handle the value estimation problem in high-dimensional action spaces.

4.   We conduct extensive experiments in a SAGIN simulation environment, employing multi-random seed statistical validation (n≥10). Comprehensive ablation studies isolate the contributions of each component. Cross-layer offloading decision analysis verifies the effectiveness of the method in selecting among space, air, and ground layers.

5.   We provide complete configuration details for the space layer (LEO satellite orbital parameters, inter-satellite links), air layer (UAV cruise parameters, battery models), and ground layer (edge servers, sensing devices) to ensure reproducibility.

To make the connection between the above challenges and the proposed design explicit, DRL-RA is organized around four corresponding mechanisms. First, the D3QN-based decision module addresses the cross-layer decision complexity by decomposing state value and action advantage for discrete choices among local, terrestrial edge, UAV, and satellite execution targets. Second, the reliability-aware modeling module separately characterizes link reliability, node availability, and delay-related reliability, and then integrates them into a path-level reliability measure for heterogeneous SAGIN links. Third, the CMDP state and action design incorporate queue states, node loads, link estimates, satellite visibility windows, and UAV availability, so that dynamic resource availability can be reflected in each decision epoch. Finally, the smooth reliability cost, softplus penalty, and Lagrangian update provide a tractable surrogate for reliability-constrained optimization, while the ablation and constraint-budget results further verify the contribution of these components.

2  Related Work

2.1 Space-Air-Ground Integrated Networks

As the core architecture of 6G, the Space-Air-Ground Integrated Network has received extensive attention from both academia and industry in recent years. Zhang et al. [1] first systematically described the technical architecture and key challenges of 6G SAGIN, pointing out that cross-layer resource management and mobility support are core difficulties. Xiao et al. [12] provided a comprehensive survey of SAGIN for 6G, systematically analyzing the architectural evolution, key enabling technologies, and cross-layer resource management challenges across space, air, and ground segments. Liu et al. [2] proposed a hierarchical architecture design for SAGIN and analyzed the coordination mechanisms among the space, air, and ground segments.

In terms of task offloading, Zhang et al. [13] studied computation offloading issues in SAGIN but did not fully consider reliability constraints. Zhang et al. [14] proposed a deep learning-based resource allocation method for SAGIN, but assumed perfect channel state information.

Liu et al. [15] proposed an online computation offloading framework for collaborative space/aerial-aided edge computing toward 6G systems, addressing the heterogeneous resource constraints across space and aerial layers. Zhang et al. [16] investigated energy-efficient computation peer offloading in satellite edge computing networks, jointly optimizing offloading decisions and power allocation under limited on-board processing capabilities and inter-satellite coordination constraints.

Huang et al. [5] further studied joint offloading and resource allocation for hybrid cloud-edge computing in SAGINs with hybrid action spaces, while Liu et al. [6] modeled edge-computing offloading in SAGIN from a game-theoretic perspective. These studies enrich the recent offloading literature, but reliability-aware redundancy and smooth reliability-cost control remain less explored.

The rapid development of LEO satellite constellations has made on-board computing feasible. Zhang et al. [16] analyzed the edge computing potential of satellite edge computing networks and investigated energy-efficient peer offloading strategies. The main challenges for satellite computing include limited on-board resources, dynamic topologies, and restricted visibility windows [17].

UAVs have unique advantages as mobile edge computing platforms. Mozaffari et al. [18] systematically reviewed the challenges and opportunities of UAV communication and computation. UAV position optimization and trajectory design have become research hotspots [19], and Yan et al. [20] employed deep reinforcement learning to optimize task offloading in UAV-assisted Internet of Vehicles, jointly optimizing UAV trajectory and resource allocation, but these works usually assume static task loads and do not consider dynamic task arrivals and reliability constraints.

2.2 Task Offloading in Edge Computing

Task offloading in edge computing environments has been extensively studied over the past decade. The fundamental tradeoff between local execution and remote offloading has been extensively studied in mobile cloud computing and edge computing literature [21,22]. Mao et al. [8] subsequently introduced Lyapunov optimization for dynamic computation offloading, achieving near-optimal performance while ensuring queue stability. Their method dynamically adjusts offloading decisions based on queue backlogs and channel conditions, providing theoretical bounds for average delay and energy consumption.

Game-theoretic methods are widely adopted in distributed task offloading where multiple users compete for limited edge resources. Chen [9] modeled the multi-user offloading problem as a potential game, proved the existence of Nash equilibrium, and designed a distributed algorithm to reach the equilibrium state. Building on this line of work, Chen et al. [23] proposed an efficient multi-user computation offloading framework for mobile-edge cloud computing that achieves Nash equilibrium through a distributed potential game formulation. Although game-theoretic approaches enable decentralized decision-making, they usually require an iterative convergence process that may not be suitable for delay-sensitive urban sensing applications.

Due to the battery constraints of mobile devices, energy-efficient offloading has received particular attention. Zhang et al. [10] proposed a joint optimization framework for energy-efficient task offloading under energy and latency constraints, considering both energy and deadline constraints. Their work showed that intelligent offloading decisions can significantly extend device lifetime while satisfying application requirements. However, these optimization methods usually assume known or predictable energy arrival patterns, which may not hold in practical urban sensing scenarios with unpredictable environmental conditions.

Geng et al. [11] proposed a DRL-based distributed computation offloading framework for vehicular edge computing, formulating the problem as an MDP and leveraging deep deterministic policy gradient to minimize system latency and energy consumption in a fully distributed manner. Xie et al. [24] jointly optimized computation offloading, resource allocation, and distributed edge service caching for multi-user MEC systems using DRL, addressing the interplay between offloading decisions and caching placement.

2.3 Deep Reinforcement Learning in Resource Management

The application of deep reinforcement learning in edge computing resource management has emerged as a prominent research direction. Mnih et al. [25] demonstrated the effectiveness of Deep Q-Networks (DQN) in learning complex control policies, inspiring subsequent applications to the task offloading problem. Chen et al. [26] proposed a distributed DRL offloading framework where each device maintains an independent DQN agent, learning to make offloading decisions based on local observations. Their method requires minimal coordination overhead while significantly outperforming heuristic baselines.

Cai et al. [3] introduced a graph-based deep reinforcement learning approach for dynamic resource allocation in SAGIN, modeling the heterogeneous network topology as a graph and leveraging spatial attention mechanisms to capture cross-layer dependencies. Li et al. [4] proposed a multi-agent reinforcement learning framework based on the centralized training with decentralized execution (CTDE) paradigm for computation offloading and resource allocation in LEO satellite edge computing networks, considering satellite mobility, dynamic topology, and heterogeneous resource constraints.

To address the challenge of limited training data and dynamic environments, Zhang et al. [22] proposed a digital twin-driven intelligent task offloading framework for collaborative MEC, leveraging digital twin technology to create virtual replicas of the physical system for real-time state monitoring and predictive offloading decisions. Zhu et al. [27] proposed ECO-SDIoT, a DRL-based edge computing offloading algorithm for software-defined IoT, utilizing SDN to provide global state information for making offloading decisions across edge and cloud resources.

Li et al. (2023) proposed TapFinger, utilizing MAPPO for decomposed scheduling and resource allocation, employing conflict resolution and action masking to achieve stability. Their work demonstrated the importance of handling infeasible actions through masking, a technique related to our reliability-aware design. HIPPO-MAT [28] emphasized decentralized execution and practical routing conflict handling, reporting extensive testbed validation. MA-CDMP [29] introduced model-based planning and mean-field conditioning, providing formal bounds for constraint satisfaction. Jia et al. [30] proposed MASTD3, a multi-agent DRL framework for real-time proportional computation offloading in industrial IoT edge computing environments, addressing device selfishness and limited bandwidth challenges. Shao et al. [31] investigated DRL-based resource management for UAV-assisted MEC under jamming attacks, jointly optimizing hover point selection, power control, and computing resource allocation while adapting to dynamic interference. Our work differs by focusing on explicit reliability modeling and smooth proxies instead of hard constraints, and adopting a Centralized Training-Distributed Execution (CTDE) paradigm with reliability guarantees.

2.4 Constrained and Safe Reinforcement Learning

Constrained reinforcement learning is becoming increasingly critical for safety-critical applications. Achiam et al. [32] proposed Constrained Policy Optimization (CPO), which provides guaranteed policy improvement while satisfying cost constraints. Tessler et al. [33] introduced Reward Constrained Policy Optimization (RCPO), combining Lagrangian methods with trust region optimization. FOCOPS [34] provides faster convergence for constrained policy optimization through first-order methods.

Wachi et al. [7] provided a comprehensive survey of constraint formulations in safe reinforcement learning, systematically classifying approaches including CMDP, chance constraints, and state-wise constraints, and offering theoretical guidance for designing reliability-constrained optimization frameworks.

A key issue in constrained RL is the choice of constraint representation. Hard indicator functions lead to sparse gradients and unstable training [32]. Smooth proxy functions, such as sigmoid-based approximations, have been proposed to resolve this issue [33]. Our work adopts and extends these techniques for reliability constraints in task offloading.

2.5 Reliability in Distributed Computing Systems

Reliability engineering has a long history in distributed computing systems, tracing back to early work on fault-tolerant distributed systems [35]. Traditional approaches to improving reliability include redundancy [36], checkpointing [37], and failure detection [38]. In the context of edge computing, reliability issues have gained renewed attention due to the mission-critical nature of many IoT applications.

Recent studies have begun incorporating reliability considerations into task offloading decisions. Xue et al. [39] proposed a DRL-based joint service caching and computation offloading scheme for vehicular edge computing systems, effectively leveraging edge resources to reduce service latency through intelligent caching and offloading coordination. Wang et al. [40] proposed a partial computation offloading approach using dynamic voltage scaling for mobile edge computing, jointly optimizing the offloading ratio and computational speed to balance energy consumption and execution latency.

Li et al. [36] investigated energy-efficient reliability-aware offloading for delay-sensitive tasks in collaborative edge computing, jointly optimizing offloading decisions, power allocation, and computation resources while ensuring task completion probability. Cao et al. [35] proposed a reliability-aware personalized deployment framework for approximate computation IoT applications in serverless mobile edge computing, considering user-specific reliability requirements and computation accuracy tradeoffs to maximize QoS. However, these methods typically rely on traditional optimization techniques with high computational complexity or use hard constraints that result in training instability.

To make the above distinction explicit, Table 1 compares the relevant research families by topology scope, constraint treatment, and remaining limitation. The comparison is used to position this work as a reliability-aware SAGIN offloading framework that combines heterogeneous space–air–ground observability, smooth reliability-cost learning, and redundancy-aware decision support rather than optimizing a single terrestrial MEC layer or using only hard feasibility filters.

3  System Model and Problem Definition

3.1 Space-Air-Ground Integrated Network Architecture

We consider a Space-Air-Ground Integrated Network (SAGIN) architecture for distributed urban sensing systems, as shown in Fig. 1. This architecture consists of three heterogeneous layers that achieve reliable task offloading and resource allocation through cross-layer coordination:

Figure 1: System architecture for SAGIN task offloading in urban sensing.

Space Layer: Consists of a Low Earth Orbit (LEO) satellite constellation, denoted as 𝒮={s1,s2,…,sNS}, with an orbital altitude of 500–2000 km. Each satellite s∈𝒮 is equipped with an on-board processing unit with computing capacity Fs∈[1,5] GHz, capable of processing compute-intensive tasks. Satellites are interconnected via Inter-Satellite Links (ISL), forming a space mesh network. The space layer provides wide-area coverage and massive data backhaul capabilities, suitable for scenarios such as large-scale environmental monitoring and disaster emergency response.

Air Layer: Consists of a Unmanned Aerial Vehicle (UAV) swarm, denoted as 𝒰={u1,u2,…,uNU}, with a flight altitude of 100–500 m. Each UAV u∈𝒰 is equipped with an edge computing module with computing capacity Fu∈[0.5,2] GHz, offering flexible near-ground computing services. UAVs possess high mobility and can dynamically adjust their positions based on task demands, suitable for temporary coverage enhancement and hotspot region services. A UAV is called visible to device d when the device–UAV distance, elevation angle, and residual UAV energy jointly satisfy the communication and serviceability conditions used in Eqs. (3), (4) and (26); otherwise, the UAV is removed from the feasible action set at that decision epoch.

Ground Layer: Consists of terrestrial sensing devices and edge servers. The set of sensing devices is denoted as 𝒟={d1,d2,…,dND}, including heterogeneous devices such as environmental sensors, traffic cameras, and smart meters. Edge servers are deployed at base stations and roadside units, denoted as ℰ={e1,e2,…,eNE}, with computing capacity Fe∈[10,50] GHz, providing low-latency local computing services.

The following problem formulation is considered over a short SAGIN scheduling horizon with periodically updated observations. The LEO visible satellite set 𝒱(t) and the remaining visibility window are estimated from predictable ephemeris information. UAVs follow planned patrol or service trajectories and periodically broadcast their position and residual energy; sudden return or failure is reflected through the availability factor ϕu(t) and feasible-action filtering rather than through a separate emergency-control process. Edge congestion is represented by queue lengths and capacity constraints, while link states, node loads, and visibility windows are updated through beacon measurements, broadcasts, and ephemeris prediction with bounded intervals. Accordingly, rare constellation-level topology mutation, simultaneous UAV-fleet failure, or regional common-cause disruptions are not modeled as separate state-transition processes; the reliability terms in Section 3.5.3 are interpreted as conditional estimates under the observed SAGIN state.

For clarity, the considered scenario is valid under four operational assumptions: (i) LEO ephemeris and satellite visibility can be predicted within the short scheduling horizon; (ii) UAVs follow planned service trajectories and report position/energy states periodically; (iii) edge congestion is represented by queue length and residual computing capacity rather than by irreversible server collapse; and (iv) link and node states are observable through bounded-interval updates. These assumptions are consistent with the simulation scope and do not imply robustness to arbitrary emergency topology mutation.

For readability, the main notation used in the problem formulation is summarized in Table 2.

3.2 Cross-Layer Task Offloading Model

3.2.1 Task Model

Each sensing device d∈𝒟 generates computational tasks according to a stochastic process. A task τd,k generated by device d at time slot k is characterized by a tuple τd,k=(Dd,k,Cd,k,Td,kmax,ρd,kmin), where Dd,k denotes the input data size (bits), Cd,k denotes the required number of CPU cycles, Td,kmax denotes the maximum tolerable delay, and ρd,kmin denotes the minimum reliability requirement.

3.2.2 Task Offloading Decisions

Under the SAGIN architecture, tasks have multiple offloading targets to choose from. We define the offloading decision variable ad∈𝒜, where the action space 𝒜={0}∪ℰ∪𝒰∪𝒮 includes:

•   ad=0: Local execution

•   ad=e∈ℰ: Offload to terrestrial edge server e

•   ad=u∈𝒰: Offload to UAV u

•   ad=s∈𝒮: Offload to satellite s

Therefore, the action space size is |𝒜|=1+NE+NU+NS. For a typical configuration of NE=10, NU=6, and NS=3, we have |𝒜|=20.

The action ad determines the final computation node, while the transmission path is automatically optimized by the network layer based on the current state. Let 𝒫d,n be the set of feasible paths from device d to target node n, the system selects the optimal path that minimizes the transmission delay:

𝒫d,n∗=arg⁡min𝒫∈𝒫d,n∑ℓ∈𝒫Ddrℓ(1)

where rℓ is the transmission rate of link ℓ. For example, feasible paths from a device to a satellite include direct links (device → satellite) and relay links (device → UAV → satellite). The system automatically selects the path with the optimal delay under current conditions. This design decouples computation node decisions from routing decisions, reducing the action space dimensionality while allowing the network layer to optimize path selection using real-time channel information.

3.3 Heterogeneous Communication Models

Communication between different layers in SAGIN exhibits significant heterogeneous characteristics and needs to be modeled separately. The terrestrial path-loss configuration follows the 3GPP urban macrocell modeling family, the air-to-ground LoS probability follows the widely used UAV communication model, and the Earth-space attenuation component follows ITU-R P.618; these models provide reproducible reference assumptions but do not eliminate site-specific calibration errors in dense urban canyons or extreme rain events [18,41,42].

3.3.1 Device-to-Edge Server Link

Communication from terrestrial devices to edge servers uses cellular links, with a transmission rate of:

rd,eg2g=Bd,elog2⁡(1+Pd⋅hd,eN0)(2)

where the channel gain hd,e=g0⋅dd,e−α⋅|hfading|2, and the path loss exponent α∈[2,4]. Typical transmission delay: 5–50 ms.

3.3.2 Device-to-UAV Link

Communication from devices to UAVs mainly uses Line-of-Sight (LoS) air-to-ground links. Considering UAV altitude Hu and horizontal distance dd,uhor, the elevation angle is θd,u=arctan⁡(Hu/dd,uhor). The LoS link probability is:

PLoS(θ)=11+aLoS⋅exp⁡(−bLoS(θ−aLoS))(3)

where aLoS,bLoS are environment-related parameters. The comprehensive transmission rate is:

rd,ug2a=Bd,u[PLoS⋅log2⁡(1+γLoS)+(1−PLoS)⋅log2⁡(1+γNLoS)](4)

where γLoS=Pd⋅|hLoS|2/N0 and γNLoS=Pd⋅|hNLoS|2/N0 are the received Signal-to-Noise Ratios (SNR) under LoS and NLoS conditions, respectively, and hLoS and hNLoS are the corresponding channel gains. Typical transmission delay: 10–100 ms, depending on elevation angle and environmental obstacles.

3.3.3 UAV-to-Satellite Link

The feeder link from UAVs to satellites uses the Ka-band, which is affected by atmospheric attenuation and rainfall. The transmission rate is:

ru,sa2s=Bu,slog2⁡(1+Pu⋅Gu⋅Gs⋅|hu,s|2N0⋅Latm)(5)

where Gu,Gs are the antenna gains of the UAV and satellite, respectively, and Latm is the atmospheric loss factor. Typical transmission delay: 20–200 ms.

3.3.4 Device-to-Satellite Direct Link

Some tasks can be directly uploaded from devices to satellites using IoT satellite communication protocols (such as satellite versions of LoRa, NB-IoT). The transmission rate is:

rd,sg2s=Bd,s⋅ηmod⋅(1−BERd,s)(6)

where ηmod is the modulation efficiency, and BERd,s is the Bit Error Rate. Typical transmission delay: 100–500 ms.

3.3.5 Total Cross-Layer Link Delay

For tasks offloaded across multiple layers, the total transmission delay is the sum of the delay of each link segment, and must also consider queuing delays and interference effects on multi-hop paths.

The basic transmission delay for device d passing through UAV u to satellite s is:

td,svia_uav=Ddrd,ug2a+Ddru,sa2s(7)

For multi-hop paths, intermediate nodes (like UAVs) need to process forwarded tasks from multiple devices, generating additional queuing delays. Let the task arrival rate when UAV u acts as a relay node be λurelay, and the service rate be μurelay, then the relay queuing delay is estimated using an M/M/1 model:

turelay_queue=1μurelay−λurelay(8)

where μurelay=ru,sa2s/D¯ is the relay service rate of the UAV-to-satellite link, and D¯ is the average task data size.

In multi-hop paths, adjacent links may experience interference. Especially when multiple UAVs transmit to the same satellite simultaneously, multiple access interference occurs. We adopt a simplified interference model, incorporating the interference effect into an effective transmission rate:

ru,seff=ru,sa2s⋅(1−αint⋅NuactiveNumax)(9)

where Nuactive is the number of concurrently transmitting UAVs, Numax is the maximum concurrency supported by the system, and αint∈[0,0.3] is the interference coefficient. As Nuactive increases, the effective transmission rate drops, and the delay increases correspondingly.

The comprehensive total delay for a multi-hop path considering queuing and interference is:

Td,smultihop=Ddrd,ug2a+turelay_queue+Ddru,seff+tsqueue+tscomp(10)

The aforementioned multi-hop delay model is used in experiments to more accurately estimate the actual delay in satellite offloading scenarios. Because queuing delay and interference effects are stochastic, we use expectation values for estimation. In high-load scenarios, these factors may cause the actual delay of satellite offloading to be significantly higher than estimates considering only transmission delay.

3.4 Hierarchical Computing Models

3.4.1 Local Execution

Delay and energy consumption for local execution:

Tdlocal=Cdfdlocal,Edlocal=κd⋅(fdlocal)2⋅Cd(11)

where fdlocal is the device CPU frequency, and κd is the energy consumption coefficient.

3.4.2 Remote Execution Unified Model

When a task is offloaded to a remote node n∈𝒜∖{0}, the total delay and energy consumption are uniformly modeled as follows.

The transmission delay using the optimal path 𝒫d,n∗:

td,ntrans=∑ℓ∈𝒫d,n∗Ddrℓ=min𝒫∈𝒫d,n∑ℓ∈𝒫Ddrℓ(12)

The total delay is:

Td,nremote=td,ntrans+tnqueue+tncomp+tn,drecv(13)

where tnqueue is the queuing delay at node n, tncomp=Cd/fd,n is the computation delay, and tn,drecv is the result return delay.

The transmission energy consumption is:

Ed,ntrans=Pdtx⋅td,ntrans=Pdtx⋅∑ℓ∈𝒫d,n∗Ddrℓ(14)

where Pdtx is the device transmission power. For multi-hop paths, the device only needs to send data once, and transmission energy depends on the total transmission time.

Wait and receive energy consumption is:

Ed,nwait=Pdidle⋅(tnqueue+tncomp),Ed,nrecv=Pdrx⋅tn,drecv(15)

Total energy consumption is:

Ed,nremote=Ed,ntrans+Ed,nwait+Ed,nrecv(16)

3.4.3 Edge Server Execution

When offloading to edge server e, the feasible path set is 𝒫d,e={Direct Link}, and transmission delay is:

td,etrans=Ddrd,eg2g(17)

Queuing delay is modeled using an M/M/1 model:

E[tequeue]=1μe−λe,μe=FeC¯(18)

3.4.4 UAV Execution

When offloading to UAV u, the feasible path set is 𝒫d,u={Device→UAV Direct Link}, and transmission delay is:

td,utrans=Ddrd,ug2a(19)

UAVs possess mobility, and their positions change over time, resulting in dynamic changes in link delay. Let the position of UAV u at time slot t be pu(t)=[xu(t),yu(t),Hu], then the distance from device to UAV is:

dd,u(t)=(xd−xu(t))2+(yd−yu(t))2+Hu2(20)

3.4.5 Satellite Execution

When offloading to satellite s, the feasible path set includes both direct and relay options:

𝒫d,s={{d→s}⏟Direct Path,{d→u→s:u∈𝒰visible}⏟Relay Path}(21)

where 𝒰visible is the current set of visible UAVs. Optimal path transmission delay:

td,strans=min{Ddrd,sg2s,minu∈𝒰visible(Ddrd,ug2a+Ddru,sa2s)}(22)

Satellites have limited computing capabilities but can process massive tasks in parallel, and their queuing delay follows an M/G/1 model. Additionally, satellite motion results in limited coverage windows, requiring tasks to be completed within the satellite’s visibility window.

3.5 Cross-Layer Reliability Model

In SAGIN, reliability is influenced by various factors, including link quality, node availability, and cross-layer transmission risks. The composite reliability model below uses a conditional-independence approximation: after conditioning on the current observed state, residual link failures, target-node availability, and delay-deadline satisfaction are treated as separable components. This approximation is common in reliability block diagrams and path-success modeling, but it can be optimistic when unobserved common causes, such as regional weather or shared power faults, affect multiple components simultaneously.

3.5.1 Link Reliability

The reliability modeling for various links is as follows.

Terrestrial link reliability is:

ψd,eg2g=1−Q(2γd,eγth)(23)

where Q(⋅) is the Q-function, and γth is the demodulation threshold SNR.

Comprehensively considering LoS and NLoS conditions, air-to-ground link reliability is:

ψd,ug2a=PLoS⋅ψLoS+(1−PLoS)⋅ψNLoS(24)

where ψLoS=1−Q(2γLoS/γth) and ψNLoS=1−Q(2γNLoS/γth) are link reliabilities under LoS and NLoS conditions, respectively.

Considering rain attenuation and atmospheric scintillation, satellite-to-ground/satellite-to-air link reliability is:

ψsat_link=exp⁡(−ArainAth)⋅ψscint(25)

where Arain is rain attenuation, and ψscint is the scintillation factor.

3.5.2 Node Availability

Edge server availability modeling adopts a two-state Markov model, with availability ϕe=μe/(λe+μe), and a failure rate λe∈[10−5,10−4]/hour.

UAV availability modeling considers battery constraints and mechanical failures, with availability:

ϕu(t)=ϕumech⋅I[Euremain(t)Eutotal>θbat](26)

where ϕumech is mechanical reliability, Euremain(t) is remaining power, and θbat is the minimum power threshold.

Considering on-board resource constraints and orbital switching, satellite availability is modeled as:

ϕs(t)=ϕshw⋅I[s∈𝒱(t)](27)

where 𝒱(t) is the set of visible satellites at time t.

3.5.3 Composite Reliability Metric

The overall reliability of offloading a task from device d to target node n∈𝒜∖{0} is calculated using the link reliabilities on the optimal path 𝒫d,n∗:

ℛd,n=Rd,ndelay⋅ϕn⋅∏ℓ∈𝒫d,n∗ψℓ(28)

where 𝒫d,n∗ is the aforementioned optimal transmission path, ψℓ is the reliability of link ℓ, ϕn is the availability of the target node, and Rd,ndelay is the delay reliability proxy:

Rd,ndelay=σ(1−E[Td,n]/Tdmaxτsmooth)(29)

where σ(⋅) is the sigmoid function, and τsmooth is the smoothing parameter. The normalized slack 1−E[Td,n]/Tdmax is positive when the expected delay is below the deadline and negative otherwise. This dimensionless expression corrects the scale of the delay slack and keeps the analytical formula consistent with the reliability proxy used throughout the framework. The sigmoid is chosen because it is a differentiable relaxation of the hard event I[Td,n≤Tdmax] and has a probabilistic interpretation: if the residual delay uncertainty around E[Td,n] follows a zero-mean logistic distribution with scale proportional to τsmoothTdmax, Eq. (29) equals the probability of meeting the deadline. A smaller τsmooth makes the proxy closer to a binary indicator but increases gradient saturation, while a larger value improves smoothness but weakens the distinction between feasible and infeasible delays.

For satellite offloading, reliability calculation is uniformly based on the delay-optimal path 𝒫d,s∗. Let the intermediate node selected by the delay-optimal path be u∗=arg⁡minu∈𝒰visible(Ddrd,ug2a+Ddru,sa2s), then:

ℛd,s=Rd,sdelay⋅ϕs⋅{ψd,sg2s,if delay-optimal path is directψd,u∗g2a⋅ψu∗,sa2s,if delay-optimal path is via UAV relay(30)

For local execution, the link-product term is omitted and the reliability reduces to the delay component and device-side execution availability. For redundant execution, the single-path reliability above is used as the per-replica success probability in Section 4.2.3. When multiple replicas share hidden common failure sources, Eq. (30) may overestimate the true success probability; the implementation mitigates this by preferring different layers and physical locations for replicas, while a full correlated-failure model is left as future work.

3.6 Constrained Markov Decision Process Definition

We model the SAGIN task offloading problem as a Constrained Markov Decision Process (CMDP) ⟨𝒮,𝒜,P,R,C,γ⟩:

•   State Space 𝒮: The state vector sd,k contains six types of information: task features, device status, status of nodes across layers, cross-layer link conditions, reliability metrics, and satellite visibility windows.

•   Action Space 𝒜: ad∈{0}∪ℰ∪𝒰∪𝒮, encompassing local execution and cross-layer offloading choices.

•   Transition Probability P: Determined by task arrivals, UAV mobility, satellite orbital motion, and link dynamics.

•   Reward Function R: rd=−ω1TdTdmax−ω2EdEdmax.

•   Cost Function C: cd=softplus(ρdmin−ℛd,ad) (smooth reliability violation).

•   Discount Factor γ∈(0,1).

Optimization Objective: Find a policy π∗ that maximizes expected cumulative reward while satisfying the expected cumulative cost constraint:

π∗=arg⁡maxπEπ[∑t=0∞γtrt]s.t.Eπ[∑t=0∞γtct]≤d(31)

where d is the cost budget, indicating the maximum tolerable reliability violation.

Execution Mode and Observability Assumptions

This paper adopts a single-agent Centralized Training-Distributed Execution (CTDE) paradigm. Specific assumptions are as follows.

In the training phase, a centralized trainer has access to global state information, including: (1) task queue status of all devices; (2) loads and availability of all edge servers, UAVs, and satellites; (3) channel state information for cross-layer links; (4) satellite orbital positions and visibility window predictions. This information is used to train a unified policy network π(a|s;θ).

In the execution phase, each sensing device d independently runs an identical copy of the policy network. The local observation od of device d is a subset of the global state s:

od=[τd,Local queue state,Neighbor node state,Link quality estimate,Satellite window](32)

where “Neighbor node state” includes edge servers and UAVs within device d’s communication range, and “Link quality estimate” is based on periodic beacon measurements.

Task characteristics τd and local queue states are maintained locally by the device. Edge server/UAV loads are obtained through periodic broadcasts. Satellite visibility windows are calculated via ephemeris prediction. Link qualities are estimated through beacon measurements and channel models. During execution, the observation dimensionality is approximately 30%–40% of the global state, yet key decision information (task features, target node loads, link reliability estimates) is obtainable.

Because all devices use the same policy network and a similar observation structure, decision behaviors maintain consistency. In practice, we add device positional encoding during training to enable the policy to adapt to devices at different locations.

4  Proposed DRL-RA Framework

This section details the proposed Deep Reinforcement Learning and Reliability-Aware (DRL-RA) framework. This framework consists of two collaborative components: (1) a D3QN-based task offloading decision module (Component A), and (2) a Reliability-Aware Multi-Objective Optimization Framework (Component B).

The two components are coupled as a decision-and-verification pipeline rather than as separate post-processing blocks: Fig. 2 shows how task features, node/link states, and visibility information feed the D3QN decision module and the RA-MOOF reliability evaluator, while Fig. 3 details the execution order from state observation and action selection to cost evaluation, Lagrangian update, and optional redundancy activation.

Figure 2: Overall architecture of DRL-RA. The input state includes task features, node load, link reliability estimates, and satellite/UAV visibility; the D3QN module selects a discrete offloading action, and RA-MOOF converts reliability estimates into a smooth cost, Lagrangian penalty, and optional redundancy decision. The arrows describe implemented data dependencies: D3QN is invoked once for each task-arrival decision, RA-MOOF evaluates the selected assignment at the same decision epoch, and redundancy is activated only after reliability and capacity checks.

Figure 3: Methodological flowchart of the proposed DRL-RA framework. The flow summarizes state observation, D3QN-based task offloading, link/node/delay reliability evaluation, capacity-gated redundancy planning, latency-aware reward and constraint-cost construction, resource reservation/release, queue update, and Q-network learning.

4.1 Component A: D3QN-Based Decision Module

4.1.1 Dueling Network Architecture

The core of Component A is the Dueling Double Deep Q-Network (D3QN). The Q-function is decomposed as:

Q(s,a;θ)=V(s;θV)+(A(s,a;θA)−1|𝒜|∑a′A(s,a′;θA))(33)

where V(s) represents the state value function, and A(s,a) represents the advantage function. This architecture is particularly suitable when action values are similar across many states.

4.1.2 Double Q-Learning

To resolve overestimation bias, we employ double Q-learning:

yk=rk+γQtarget(sk+1,arg⁡maxa′Qmain(sk+1,a′);θ−)(34)

4.1.3 Action Space Design

Action ai∈{0,1,…,M} represents a categorical choice rather than fractional allocation. For categorical decisions, a DQN designed for discrete actions is appropriate; since we do not perform partial offloading, Dirichlet distributions are unnecessary.

4.2 Component B: Reliability-Aware Multi-Objective Optimization Framework

4.2.1 Smooth Reliability Proxy

A key contribution of this paper is utilizing a smooth reliability proxy instead of hard indicator functions. The reliability constraint cost is calculated as:

cirel=softplus(ρimin−ℛi,ai)(35)

where softplus(x)=log⁡(1+ex) provides a smooth approximation of max(0,x). This surrogate does not claim exact preservation of hard per-task chance constraints. It defines a smooth CMDP cost whose expectation is optimized by the Lagrangian update. Because max(0,x)≤softplus(x) for all x, satisfying a budget on the softplus cost is conservative with respect to the expected positive reliability deficit, but it cannot guarantee zero violation probability for every individual task.

4.2.2 Lagrangian Constraint Handling

We adopt the Lagrangian method to handle reliability constraints. The Lagrangian objective is:

ℒ(π,λ)=Eπ[r]−λ(Eπ[c]−d)(36)

where λ≥0 is the Lagrangian multiplier updated via projected stochastic subgradient ascent on the dual variable:

λk+1=max(0,λk+ηλ(Eπ[c]−d))(37)

4.2.3 Redundancy-Aware Backup Policy

For critical tasks where ρimin>ρth, we implement redundancy:

nireplica=min(⌈log⁡(1−ρimin)log⁡(1−ℛ¯i)⌉,nmax)(38)

Eq. (38) follows from the reliability of parallel independent replicas. If each replica succeeds with average probability ℛ¯i, the probability that all n replicas fail is (1−ℛ¯i)n, and the probability that at least one replica succeeds is 1−(1−ℛ¯i)n. Requiring 1−(1−ℛ¯i)n≥ρimin gives n≥log⁡(1−ρimin)/log⁡(1−ℛ¯i) because log⁡(1−ℛ¯i)<0. Replicas are distributed across different servers to minimize correlated failures. This derivation therefore uses approximate conditional independence among replica success events rather than assuming that all physical failures in SAGIN are uncorrelated. In deployment, replicas are preferentially placed on different layers, links, or physical regions to reduce shared-risk exposure. If multiple replicas share the same weather-affected air-to-ground channel, backhaul link, or regional interference source, Eq. (38) may become optimistic; modeling such common-cause failures through correlated-failure factors or copula-based reliability terms is left as a focused extension.

After the policy selects a primary action ai, for tasks with high reliability requirements (ρimin>ρth), the system creates replicas following these steps:

1.   Calculate the required number of replicas nireplica according to Eq. (38), ensuring overall reliability meets the requirement.

2.   Select replica targets from the available node set 𝒜∖{ai}, prioritizing nodes with high failure independence from the primary target node (e.g., different layers, different physical locations).

3.   Reserve computing resources on each replica target node nj:

Fnjreserved←Fnjreserved+CiTimax−tcurrent(39)

Resource reservation ensures replicas are not rejected due to insufficient capacity.

4.   Replica tasks are added to the target nodes’ task queues, utilizing the same priority rules as the primary task. Queue state update:

Qnj←Qnj+Dird,nj(40)

5.   The first successfully completed replica triggers a cancellation signal, terminating the execution of other replicas. The deduplication logic is implemented via tracking task IDs:

if τicompleted then cancel({τireplica,k}k=1nireplica)(41)

6.   Prior to creating replicas, the system checks the available capacity of each target node:

Fnjavailable=Fnjtotal−Fnjreserved−Fnjused(42)

If Fnjavailable is insufficient to support the replica, that node is skipped or the replica count is reduced.

To avoid double counting, the system maintains a unified resource ledger recording the resource footprint of each task. Pre-reserved resources are immediately released upon task completion or cancellation. If all feasible edge/UAV/satellite nodes lack sufficient residual capacity, the system does not create virtual resources; it reduces the effective replica count to the number of feasible targets and records the remaining reliability gap as a constraint cost. This capacity-gated rule prevents the redundancy mechanism from hiding resource-exhaustion cases in boundary scenarios.

4.3 Reliability-Constrained Integration of D3QN Decisions and RA-MOOF

Integration is achieved through a reliability-weighted reward function:

ritotal=riperf+ωrel⋅ℛi,ai⋅Ismooth[ℛi,ai≥ρimin]−ωpen⋅cirel(43)

where Ismooth is a smoothed indicator function using sigmoid. The weights ωrel and ωpen determine the reliability–latency–energy tradeoff: increasing ωrel or ωpen makes the policy more conservative and usually lowers constraint violations, while smaller values prioritize latency and energy. Therefore, the weights should be selected according to the service-level agreement of the target urban-sensing application rather than treated as universal constants.

Component A provides rapid categorical decisions through a neural-network forward pass, but reward shaping alone does not enforce reliability constraints. Component B provides explicit reliability evaluation and a smooth constraint-violation cost, but it is inefficient as a stand-alone online mixed-integer optimizer. Their integration uses D3QN for fast action selection and RA-MOOF for reliability-aware reward/cost construction; the resulting guarantee is a surrogate expected-cost guarantee under the CMDP assumptions rather than a hard per-task guarantee under arbitrary correlated failures.

During online deployment, the interaction frequency is one D3QN–RA-MOOF cycle per task-arrival decision epoch. First, the current task, queue, node-load, visibility, and link-state observations are encoded into st. Second, D3QN outputs the primary offloading action at. Third, RA-MOOF evaluates link, node, and delay reliability for that selected assignment at the same epoch and forms the smooth cost cirel. Fourth, redundancy planning is not executed for every task indiscriminately; it is triggered only when the task has a high reliability requirement or when the selected assignment leaves a positive reliability gap after capacity filtering. Finally, the task is executed, queues/resources are updated, and the Lagrange multiplier is updated over the sliding cost window W.

4.4 RA-Opt Optimization Problem Definition

RA-Opt is the pure optimization implementation of Component B and is used as an optimization reference in ablation comparisons. It solves the resource-allocation problem online at each decision epoch and is therefore not directly comparable with trained DRL policies in terms of deployment-time inference latency. In this paper, RA-Opt is positioned as a non-real-time optimization reference for assessing the quality of reliability-aware offloading decisions, rather than as a directly deployable neural-inference baseline for decision-latency comparison. Its complete optimization problem definition is as follows.

Decision variables include:

•   xi,n∈{0,1}: Whether task i is offloaded to node n.

•   fi,n≥0: CPU frequency allocated to task i (if xi,n=1).

Objective function where ω1 and ω2 weight normalized delay and normalized energy, respectively; increasing ω1 favors lower latency, while increasing ω2 favors lower energy consumption:

minx,f∑i=1N∑n∈𝒜xi,n(ω1Ti,nTimax+ω2Ei,nEimax)(44)

Constraints:

∑n∈𝒜xi,n=1,∀i(Each task is offloaded to exactly one node)Ti,n⋅xi,n≤Timax,∀i,n(Delay constraint)ℛi,n≥ρimin−ϵi,∀i,n:xi,n=1(Reliability constraint)∑i:xi,n=1fi,n≤Fnmax,∀n(Capacity constraint)xi,n∈{0,1},fi,n≥0(Variable domains)(45)

Composite reliability ℛi,n is calculated according to Eq. (28), containing link reliability ψℓ, node availability ϕn, and delay reliability proxy Ri,ndelay. Because ℛi,n involves sigmoid functions, the problem is a non-convex Mixed Integer Programming (MIP) problem.

We employ the CVXPY framework utilizing the ECOS_BB solver for MIP processing. Relaxed problems can be approximately solved through sequential convex optimization. In experimental configurations, solver tolerance is set to 10−4, with a maximum of 1000 iterations.

4.5 Complexity and Feasibility Discussion

For one decision epoch, D3QN inference requires one forward pass over |𝒜| discrete actions; with a three-layer MLP of hidden widths (256,128,64), the dominant cost is linear in the number of action scores and matrix multiplications. The Lagrangian update adds only O(1) arithmetic per update window, while redundancy checking is O(|𝒜|log⁡|𝒜|) if candidate nodes are sorted by reliability or residual capacity. By contrast, RA-Opt contains binary offloading variables and continuous CPU-allocation variables, so its worst-case mixed-integer solving cost can grow exponentially in the number of tasks and candidate nodes.

The feasibility guarantee is stated at the surrogate-CMDP level. Under bounded rewards and costs, finite action sets, and standard stochastic-approximation step-size conditions, primal-dual Lagrangian updates converge to a stationary point or neighborhood of the surrogate constrained objective in the tabular or compatible-function-approximation setting. With nonlinear neural networks, the paper does not claim global convergence; instead, feasibility is empirically checked through the expected cost E[c] and constraint violation rate in Table 8. This distinction is important because the softplus cost is a conservative smooth upper bound of the positive reliability deficit, not an exact hard reliability indicator.

As summarized in Algorithm 1, the proposed training procedure couples D3QN-based action selection with RA-MOOF reliability evaluation, optional replica activation, and Lagrangian multiplier adaptation, so that policy learning and reliability-constraint control are updated within the same episode loop.

5  Experimental Setup

5.1 SAGIN Simulation Environment

We build a SAGIN simulation environment for urban sensing, modeling a 10 km × 10 km urban area, containing full space layer, air layer, and ground layer infrastructure.

5.1.1 Space Layer Configuration

The simulation includes NS=3 LEO satellites with an orbital altitude of 550 km (similar to Starlink orbits) and an inclination of 53 degrees. Each satellite is equipped with an on-board processing unit with a compute capacity of Fs∈[2,4] GHz and 10 GB of storage. Satellites inter-connect via laser inter-satellite links with a 10 Gbps link capacity. Satellite visibility windows last approximately 10 min, and maximum elevation angles depend on user locations.

Satellite Visibility Window Management Mechanism: Due to the movement of LEO satellites, the visibility window over a specific ground area is limited (around 10 min), heavily impacting task offloading decisions. This paper handles visibility window constraints using the following mechanisms:

(1) Ephemeris Prediction: The system predicts future satellite positions and visibility at time t+Δt based on orbital parameters (altitude, inclination, right ascension of the ascending node, etc.). For the short prediction horizon used in this simulation (Δt<5 min), the orbital-propagation model provides second-level time resolution for visibility-window estimation.

(2) Visibility Check: At decision time t, the system first checks target satellite s’s visibility status I[s∈𝒱(t)], where 𝒱(t) is the set of visible satellites. Offloading to a satellite is only allowed if it is visible.

(3) Remaining Visibility Time Estimation: For currently visible satellites, the system estimates remaining visibility time Tsremain and compares it with expected task completion time Td,stotal. If Tsremain<Td,stotal+Tmargin (where Tmargin is a safety margin, typically 30 s), the satellite is deemed unselectable to prevent interruption upon window closure.

(4) Adversarial Scheduling Scenarios: Under extreme loads or emergencies, multiple high-priority tasks might compete for limited windows. Here, the system deploys redundancy backup policies, broadcasting critical tasks to multiple visible satellites to boost reliability during window handovers. This mechanism is evaluated only as a high-load visibility-window competition scenario; it is not a claim of robustness to arbitrary constellation topology mutation, which is outside the observed-state formulation described in Section 3.

5.1.2 Air Layer Configuration

The simulation deploys NU=6 UAVs cruising at altitudes of 150–300 m and speeds of 5–15 m/s, conducting fixed-point patrol missions over the city. Each UAV carries an edge module with Fu∈[0.5,1.5] GHz capacity and a 5000 mAh battery lasting around 30 min. UAV-ground communication leverages millimeter waves with a 500 MHz bandwidth.

5.1.3 Ground Layer Configuration

The ground layer contains ND=100 sensing devices and NE=10 edge servers. Device types include environmental sensors (40%), traffic cameras (25%), smart meters (20%), and public safety sensors (15%). Edge servers are placed at cellular base stations with a 1.5 km coverage radius and capacities of Fe∈[10,50] GHz.

5.1.4 Task Generation

Tasks follow a Poisson process with sinusoidally varying intensity over time to capture daily patterns. To preserve the heterogeneity of urban sensing workloads, the task-size, CPU-cycle, deadline, and reliability-requirement ranges are sampled by device category as summarized in Table 3.

5.1.5 Cross-Layer Link Parameters

Because each offloading action may traverse terrestrial, aerial, or satellite links, the simulator assigns different bandwidth, delay, and reliability ranges to each link class; these cross-layer communication assumptions are listed in Table 4 and are used when computing transmission delay and link reliability.

The heterogeneous link parameters in SAGIN are shown in Table 4.

Terrestrial link path loss follows the 3GPP Urban Macrocell model: PL(d)=128.1+37.6log10⁡(d) dB. Air-to-ground LoS parameters: a=9.61, b=0.16 (urban environment). Satellite-ground links incorporate rain attenuation via the ITU-R P.618 model.

5.1.6 Reliability Parameters

Reliability parameters across all layers:

•   Edge Servers: Failure rate λe∈[10−5,10−4]/hour, recovery rate μe∈[0.1,0.5]/h

•   UAVs: Mechanical reliability ϕumech=0.995, battery threshold θbat=20%

•   Satellites: Hardware reliability ϕshw=0.999, availability within visibility window ≈0.98

5.2 Baselines and Implementation Details

We benchmark against the following methods, providing full configuration details. Note: Because our action space is discrete, methods suited purely for continuous action spaces (like DDPG) are excluded. Recent graph-DRL and MARL methods are discussed in Table 1. They are not assigned synthetic numerical scores in Table 7 because their published settings use different observation structures, agent definitions, and optimization objectives; adding unsupported numbers would reduce rather than improve fairness.

The closest state-of-the-art family to this work consists of SAGIN/LEO offloading methods based on graph DRL and MARL. These studies are discussed as closely related work because they address dynamic heterogeneous topology, but a direct numerical entry requires the observation model, agent definition, and reliability constraint to be reproducible under the same simulator. The selected numerical baselines therefore cover the comparable algorithmic families under the same task traces: heuristic offloading, queue-aware optimization, unconstrained DRL, constrained RL, the D3QN-only component, and the RA-Opt optimization reference. The reproducible settings for these baselines are given in Table 5, which makes explicit whether each method uses no constraint handling, a queue-stability rule, a policy-gradient constraint, a Lagrangian relaxation, or the proposed smooth reliability mechanism.

We selected DQN and PPO as unconstrained DRL baselines because they accommodate discrete action spaces. We excluded DDPG because it is an actor-critic method custom-built for continuous action spaces, yielding deterministic continuous action values. Applying DDPG to a discrete action space necessitates discretizing actions or utilizing techniques like Gumbel-Softmax, which introduces extraneous approximation errors and diverges from the method’s original design intent. Therefore, to preserve fairness and methodological rigor, DDPG is omitted. To ensure a fair comparison, all DRL baselines (DQN, PPO, CPO, RCPO, Lagrangian-PPO, FOCOPS, D3QN, DRL-RA) utilize the exact same MLP [256, 128, 64] architecture. This design decision nullifies network capacity discrepancies, ensuring performance variances reflect pure algorithmic merit rather than scale differences.

For hyperparameter fairness, we separate common settings from algorithm-specific settings. The network width, training horizon, number of seeds, evaluation workload, and state/action definitions are identical across learning-based methods. Method-specific parameters, such as PPO clipping and epochs, CPO trust-region radius, entropy coefficient, replay-buffer settings, and Lagrangian update rates, are selected from standard stable ranges recommended for each algorithm and then fixed before final testing. When default settings produce unstable learning, parameters are adjusted only within the same predefined stable ranges used for baseline tuning. All final comparisons use the same ten random seeds and the same evaluation traces, so the tuning budget remains comparable across methods.

After this tuning protocol is fixed, Table 6 reports the DRL-RA-specific values used in all final experiments, including the learning rates, replay setting, constraint budget, smoothness coefficient, and redundancy trigger.

The hyperparameters are selected from standard stable ranges used in DQN and constrained-RL implementations and then fixed for all random seeds. The learning rate 10−4 is used to avoid unstable Q-value updates in the mixed reliability/latency reward; the discount factor 0.99 reflects the long-horizon effect of queueing and resource reservation; the target-network update period and replay-buffer size follow common DQN stabilization practice. The smoothness value τsmooth=0.1 is applied to normalized delay slack, so it corresponds to a transition band of roughly ten percent of the deadline.

The cost budget d=0.05 should be read as a long-term normalized smooth reliability-shortfall budget rather than as a hard per-task violation probability. Since the softplus cost is formed from the reliability gap after normalization, d=0.05 means that the expected residual shortfall is constrained to a small margin around five percent of the unit-normalized cost scale. This is appropriate for SAGIN scheduling because transient visibility and channel fluctuations make a zero-violation target unrealistic, while the separate CVR metric still reports hard task-level violations.

Decision latency measurements are conducted on an Intel Xeon E5-2680 v4 CPU (2.4 GHz) with 64 GB RAM and an NVIDIA Tesla P100 GPU. Neural network inference uses PyTorch 1.12 with no model compression or quantization applied. The batch size is set to 1 (single-task decision scenario).

5.3 Evaluation Metrics

•   Task Completion Rate (TCR): Percentage of tasks completed within deadlines.

•   Average Latency (AL): Mean end-to-end latency (ms).

•   Energy Consumption (EC): Average energy consumed per task (mJ).

•   System Reliability (SR): Average ℛi,j of completed tasks.

•   Resource Utilization (RU): Mean utilization of edge servers.

•   Decision Latency (DL): Time per decision (ms).

•   Constraint Violation Rate (CVR): Proportion of tasks failing reliability thresholds.

•   Expected Cost E[c]: Average reliability violation cost, verifying budget fulfillment.

5.4 Statistical Methodology

All outcomes are reported as means ± standard deviations across 10 random seeds. Statistical significance is evaluated with paired t-tests at p<0.05, and confidence intervals are estimated using 1000 bootstrap resamples. In the main comparison table, an asterisk marker is added to DRL-RA results that are significantly better than the strongest comparable learning-based baseline under the same evaluation traces; confidence intervals are reported in the table note for the primary metrics.

6  Experimental Results and Analysis

This section reports the empirical evaluation. We first compare overall performance, then isolate the contribution of each component through ablation studies, and finally analyze sensitivity to reliability requirements, system size, workloads, and channel conditions.

6.1 Overall Performance Comparison

6.1.1 Experiment Design

This experiment systematically benchmarks DRL-RA against standard paradigms. Employing a strict control scheme guarantees fair comparisons within uniform environments. Specific settings:

We picked 12 benchmarks spanning three categories: (1) Classic methods including Random (random target selection), Greedy (nearest edge server), and Lyapunov optimization; (2) Unconstrained DRL (DQN, PPO); (3) Constrained RL (CPO, RCPO, Lagrangian-PPO, FOCOPS). Additionally, we benchmarked D3QN (Component A only) and RA-Opt (Component B only) to validate the integration.

Each learning-based method is trained with 10 random seeds for 1000 episodes and 1000 steps per episode. After training, each policy is evaluated over 10,000 time steps. Metrics include TCR, AL, EC, SR, RU, and DL. RA-Opt is solved online and is therefore reported as a non-real-time optimization reference for decision-quality assessment rather than as an inference-latency peer.

Results are reported in mean ± SD format. Significance is assessed via paired t-tests (p<0.05), and 95% confidence intervals and Cohen’s d are used to quantify effect size.

The overall comparison in Table 7 is used as the basis for the subsequent analysis because it reports not only task completion and latency, but also energy consumption, reliability, resource utilization, and decision latency under the same traces. This multi-metric view is necessary to distinguish a genuine reliability-aware gain from a policy that merely shifts cost to energy use or online computation time.

6.1.2 Result Analysis

Using the six metrics in Table 7, the analysis below first evaluates task completion and latency, then examines whether the gain is achieved without sacrificing energy, reliability, resource utilization, or deployment-time inference speed.

Table 7 illustrates performance comparisons across six core metrics.

DRL-RA achieves a TCR of 95.6±0.8%, which is the highest value among the considered baselines. Compared with Lagrangian-PPO (91.8±1.0%), the absolute improvement is 3.8 percentage points and is statistically significant (p<0.001, Cohen’s d=4.2). The lower standard deviation of DRL-RA also indicates more stable behavior across random seeds. The comparison between constrained methods and unconstrained DQN/PPO supports the need for explicit reliability-cost modeling, while the additional gain of DRL-RA over Lagrangian-PPO indicates the benefit of combining the smooth reliability proxy with redundancy backup.

For average latency, DRL-RA obtains 712±42 ms, which is 11.1% lower than Lagrangian-PPO (801±48 ms). This result is consistent with the role of the dueling architecture in separating state-value and action-advantage estimation, which is useful when several offloading targets have similar action values. The smooth reliability term also avoids the abrupt policy changes that can occur with binary constraint indicators, allowing the policy to trade latency against reliability margins more gradually.

DRL-RA obtains the lowest energy consumption among the reported methods (144±8 mJ). The reduction is consistent with fewer unnecessary long-haul transmissions and better load distribution across feasible edge, UAV, and satellite targets. Lyapunov optimization remains a useful queue-stability reference, but its 15.7±2.1 ms online decision latency is higher than that of neural inference policies under the same hardware setting.

For system reliability, DRL-RA achieves 96.2±0.7%, which is 7.6 percentage points higher than D3QN and 3.9 percentage points higher than Lagrangian-PPO. This result supports the contribution of RA-MOOF because the smooth proxy, Lagrangian update, and redundancy policy each target a different source of reliability degradation: marginal delay violations, long-term budget drift, and high-reliability task failure.

DRL-RA obtains the highest resource utilization (81.5±2.4%) while maintaining the highest reliability. This indicates that the policy does not simply avoid loaded nodes; instead, it uses reliability-aware load distribution to exploit available resources while keeping the reliability cost within the budget.

Decision latency should be interpreted with the solver type in mind. DRL-RA requires 3.7±0.4 ms for one neural-policy inference, while RA-Opt requires 42.6±5.3 ms because it solves a mixed-integer optimization problem online. Thus, the RA-Opt latency value reflects online optimization cost, whereas deployment-time latency comparisons are primarily made among trained inference methods.

6.1.3 Constraint Budget Fulfillment Verification

To connect the surrogate feasibility discussion in Section 4.5 with empirical behavior, Table 8 compares the expected reliability cost E[c] with the budget d and also reports CVR for each constrained or unconstrained learning method.

To attest to CMDP constraint compliance, we profile expected costs E[c] vs. budget d across constrained RL schemes. Table 8 displays thorough constraint satisfaction stats.

The expected cost of DRL-RA is E[c]=0.043±0.005, which is below the budget d=0.05. This verifies satisfaction of the surrogate expected-cost constraint used in Eq. (31). Unconstrained DQN and PPO exceed the cost budget, while the constrained variants satisfy or nearly satisfy it; among them, DRL-RA has the lowest expected cost and the lowest CVR in Table 8.

6.1.4 Synergistic Effect between Components A and B

The comparison among D3QN, RA-Opt, and DRL-RA indicates complementary behavior. D3QN has low neural-inference latency (2.2±0.3 ms) but lower reliability (88.6±1.5%). RA-Opt provides an optimization-reference reliability of 91.2±1.2%, while its 42.6±5.3 ms decision time reflects the cost of online mixed-integer solving rather than neural-policy inference. DRL-RA preserves millisecond-level inference latency (3.7±0.4 ms) while increasing reliability to 96.2±0.7%, supporting the design choice of using D3QN for action selection and RA-MOOF for reliability-aware cost construction.

6.2 Comprehensive Ablation Experiments

6.2.1 Experiment Design

The ablation experiments evaluate the contribution of each DRL-RA module by removing or replacing one component at a time and reporting the resulting performance change. This single-factor design helps attribute the observed differences to the corresponding module while keeping the remaining training and evaluation settings unchanged.

The ablation table is organized to test this attribution logic directly: Table 9 groups removals by the D3QN decision module, the RA-MOOF reliability module, and their reward-level integration, so each performance drop can be linked to a specific design choice.

Ablation Configurations: We structured 12 setups across three tiers:

(1) Component A (D3QN) Ablation: Evaluates the Dueling architecture and Double Q-learning. The configurations remove Dueling while keeping Double Q-learning, remove Double Q-learning while keeping Dueling, or remove both to obtain vanilla DQN.

(2) Component B (RA-MOOF) Ablation: Evaluates the reliability-aware components. The configurations remove RA-MOOF entirely, replace the smooth proxy with a hard indicator, replace the adaptive Lagrangian multiplier with a fixed multiplier, or remove redundancy backup.

(3) Integration Ablation: Evaluates the interaction between the decision module and the reliability-aware objective by removing the reliability reward or the reliability penalty.

Metrics: Each configuration is evaluated over 10 independent seeds, and TCR, AL, and SR are reported. Performance changes are measured relative to the full DRL-RA configuration.

6.2.2 Component A (D3QN) Ablation Analysis

Removing the Dueling architecture reduces TCR from 95.6% to 93.9% (−1.7 percentage points), increases average latency from 712 to 789 ms (+10.8%), and reduces SR from 96.2% to 94.5% (−1.7 percentage points). This result is consistent with the role of Dueling networks in separating the state-value term V(s) from the action-advantage term A(s,a). In SAGIN offloading, several candidate nodes can have similar latency but different reliability or queue states; separating value and advantage helps distinguish such actions more stably.

Removing Double Q-learning decreases TCR from 95.6% to 94.2% (−1.4 percentage points), increases latency from 712 to 757 ms (+6.3%), and decreases SR from 96.2% to 95.3% (−0.9 percentage points). Double Q-learning decouples action selection from action evaluation and therefore reduces overestimation bias. In the offloading setting, reducing this bias helps avoid repeatedly selecting apparently high-value but congested nodes.

Removing both mechanisms, i.e., reverting to vanilla DQN, reduces TCR by 6.1 percentage points, increases latency by 25.3%, and reduces SR by 9.9 percentage points. The larger degradation compared with removing either mechanism alone suggests that Dueling and Double Q-learning provide complementary benefits: the former improves value decomposition, while the latter improves value-estimation stability.

6.2.3 Component B (RA-MOOF) Ablation Analysis

Replacing the smooth proxy with hard indicator functions reduces TCR from 95.6% to 92.8% (−2.8 percentage points) and SR from 96.2% to 93.4% (−2.8 percentage points). The standard deviation of SR also increases from 0.7% to 1.1%, indicating less stable training across random seeds. A hard indicator provides only sparse binary feedback, whereas the smooth proxy supplies continuous information about the degree of reliability deficit. This denser signal can reduce TD-target variance and yields smoother estimates for the Lagrangian cost term E[c].

Fixing the Lagrangian multiplier at λ=1 reduces TCR from 95.6% to 93.5% (−2.1 percentage points) and SR from 96.2% to 91.2% (−5.0 percentage points). A fixed multiplier cannot adapt to different violation levels: if it is too small, the policy under-penalizes reliability deficits; if it is too large, the policy may become overly conservative. The adaptive update adjusts the penalty according to the observed constraint cost, and in our runs λ moves from 1.0 to approximately the range [2.1, 2.5].

Removing redundancy reduces TCR from 95.6% to 94.1% (−1.5 percentage points) and SR from 96.2% to 94.9% (−1.3 percentage points). The effect is more visible for high-reliability tasks with ρmin≥0.95: their completion rate decreases from 89.2% to 81.7% without redundancy. This supports the use of capacity-gated replicas for mission-critical tasks rather than for all tasks indiscriminately.

6.2.4 Integration Ablation Analysis

Removing the reliability reward reduces TCR from 95.6% to 92.5% (−3.1 percentage points) and SR from 96.2% to 91.2% (−5.0 percentage points). This indicates that the reliability reward helps the policy prefer nodes with higher reliability margins when several actions satisfy basic feasibility conditions.

Removing the reliability penalty reduces TCR from 95.6% to 93.4% (−2.2 percentage points) and SR from 96.2% to 92.8% (−3.4 percentage points). The immediate penalty and the Lagrangian multiplier act at different levels: the former shapes step-wise decisions, while the latter regulates the long-term expected constraint cost. Their combination therefore improves both local action selection and long-term constraint compliance.

6.2.5 Ablation Summary

The ablation results show five patterns: (1) the dueling and double-Q mechanisms jointly improve the discrete decision module; (2) the smooth proxy is important for stable reliability-cost learning; (3) adaptive Lagrangian updates help align reward optimization with the cost budget; (4) redundancy mainly benefits high-reliability tasks; and (5) reward and penalty weights control the tradeoff between performance and constraint compliance.

6.3 Sensitivity Analysis

The sensitivity analysis evaluates DRL-RA under different reliability requirements, system sizes, workload intensities, and channel conditions to examine its robustness within the simulated operating range.

6.3.1 Performance under Varied Reliability Needs

Design: We compare DRL-RA with Lagrangian-PPO and D3QN across five reliability thresholds: 0.80, 0.85, 0.90, 0.95, and 0.98. TCR and CVR are reported over 10 random seeds.

Motivation: Urban sensing applications have different reliability requirements. Environmental monitoring can tolerate moderate reliability thresholds, whereas public-safety tasks may require stricter thresholds such as ρmin≥0.98. Evaluating multiple thresholds clarifies the reliability–performance tradeoff. Accordingly, Table 10 reports both TCR and CVR at each threshold, allowing the analysis to separate performance degradation caused by stricter reliability requirements from actual constraint-control behavior.

The two panels in Fig. 4 visualize the same threshold sweep from complementary perspectives: panel (a) highlights the TCR loss as reliability requirements tighten, whereas panel (b) shows whether this loss is accompanied by lower constraint violations.

Figure 4: Performance under varied reliability requirements. (a) Task completion rate across different minimum reliability thresholds; (b) Constraint violation rate across different minimum reliability thresholds.

As the reliability requirement increases, all methods show lower TCR, but the degradation rates differ. D3QN decreases to 65.9% at ρmin=0.98, a 27.6-point drop from the threshold 0.80 case, whereas DRL-RA remains at 82.5%, corresponding to a 14.3-point drop. This indicates better robustness of DRL-RA under stricter reliability requirements in the simulated setting.

DRL-RA keeps CVR in the 3%–4% range and reaches 2.7±0.9% at ρmin=0.98. D3QN shows higher violation rates across the same thresholds, while Lagrangian-PPO reduces violations but remains above DRL-RA in this simulation.

Using the TCR decrease per 0.1 increase in reliability threshold as a performance-loss ratio, DRL-RA loses 7.9 percentage points per 0.1 threshold increase, whereas D3QN loses 15.3 percentage points. This suggests that DRL-RA handles the reliability–performance tradeoff more effectively in this experiment.

At ρmin=0.98, DRL-RA completes 82.5% of tasks compared with 65.9% for D3QN. The improvement is consistent with the redundancy mechanism: the simulated system creates 1.8 replicas per high-reliability task on average at ρmin=0.98, compared with 1.1 replicas at ρmin=0.80.

6.3.2 Scalability across System Sizes

Design: We evaluate DRL-RA overhead and performance under four system sizes: (1) Small: 50 devices and 5 edge servers; (2) Medium: 100 devices and 10 edge servers; (3) Large: 200 devices and 15 edge servers; and (4) Ultra: 500 devices and 25 edge servers. Each setting is tested over 5 seeds, and TCR and DL are recorded.

The scalability curves in Fig. 5 show the expected decline in completion rate as the system expands, but they also allow the degradation of DRL-RA and D3QN to be compared under the same growth in devices and candidate offloading targets.

Figure 5: Scalability analysis (each group is configured with n=5 seeds).

Larger systems reduce TCR for all methods because resource contention and the decision space both increase. When the number of devices grows from 50 to 500, DRL-RA decreases by 8.4 points (97.1% to 88.7%), whereas D3QN decreases by 12.6 points (94.2% to 81.6%). This indicates better scalability of the reliability-aware policy in the simulated system-size range.

Decision latency scales sub-linearly in this experiment. When the system grows from 50 to 500 devices, DRL-RA’s latency increases from 3.1 to 7.3 ms (2.35×), while the action space increases from 6 to 26 actions (4.33×). This result is consistent with the batched matrix operations used in the neural-network forward pass. Even in the 500-device setting, the observed decision latency remains below 10 ms.

Training on the 500-device setting takes 3.2× longer than the medium-size baseline. The increase is mainly caused by environment interaction and experience collection rather than by the policy update itself, suggesting that parallelized data collection could further reduce training time.

6.3.3 Robustness under Workloads and Channel Noise

Design: We evaluate varying workloads and channel SNR conditions. The arrival rates are Low (5 tasks/s/device), Medium (12.5 tasks/s/device), and High (25 tasks/s/device). The SNR levels are Low (5 dB), Medium (15 dB), and High (25 dB). Each setting is evaluated over 10 seeds.

The robustness plots in Fig. 6 connect two different stress factors to the same performance metrics: workload growth mainly increases queueing pressure, whereas lower SNR directly weakens link reliability and increases transmission time.

Figure 6: Sensitivity to task arrival rate and channel SNR.

Under high arrival rates (25 tasks/s/device), queues become longer and task drop risk increases. DRL-RA obtains a 91.4% TCR, which is 5.8 percentage points lower than the low-load case. The result is consistent with load balancing across feasible nodes. Average latency increases by 35.6%, while SR decreases by 4.4 percentage points, showing that reliability is less sensitive than latency to workload growth in this simulation.

Low SNR (5 dB) reduces link reliability ψi,j and increases transmission time. DRL-RA obtains an 88.7% TCR in this setting, which is 8.4 percentage points lower than the high-SNR case. Latency increases by 55.7%, and the policy compensates by selecting local execution or alternative links when their estimated reliability is higher.

Using a robustness index defined as fluctuation range divided by baseline performance, DRL-RA obtains 5.9% under arrival-rate variation and 8.8% under SNR variation. The larger fluctuation under SNR changes is consistent with the direct effect of channel quality on link reliability and transmission time.

7  Conclusion

This paper presented DRL-RA, a reliability-aware and low-latency task offloading and resource allocation framework for SAGIN. The method combines D3QN-based discrete action selection with RA-MOOF-based reliability cost construction, smooth surrogate constraint handling, and capacity-gated redundancy for high-reliability tasks. The system model explicitly captures heterogeneous communication links, hierarchical computing nodes, satellite visibility windows, UAV availability, and composite reliability under stated conditional-independence assumptions. Simulation results across multiple random seeds show that DRL-RA improves task completion rate, latency, and system reliability over the implemented baselines, while the constraint-budget table verifies satisfaction of the surrogate expected-cost constraint. The problem formulation specifies the observation and update assumptions under which satellite visibility windows, UAV availability, queue states, and link measurements are used for repeated short-horizon decisions. Future work can further extend this formulation to explicitly model correlated large-scale disruptions.

Acknowledgement: Not applicable.

Funding Statement: The authors received no specific funding for this study.

Author Contributions: The authors confirm contribution to the paper as follows: Conceptualization, Fei Bu and Zheng Wang; methodology, Zheng Wang and Yong Pan; software, Zhaomin Wu and Yuchen Liang; validation, Fei Bu and Zhaomin Wu; formal analysis, Zheng Wang; investigation, Fei Bu, Yong Pan and Zhongshan Zhu; resources, Zhongshan Zhu; data curation, Yuchen Liang and Zhaomin Wu; writing—original draft preparation, Fei Bu and Zheng Wang; writing—review and editing, Zheng Wang and Tengfei Tu; visualization, Zhaomin Wu and Yuchen Liang; supervision, Zheng Wang and Tengfei Tu; project administration, Fei Bu; funding acquisition, Fei Bu and Zheng Wang. All authors reviewed and approved the final version of the manuscript.

Availability of Data and Materials: The data that support the findings of this study are available from the Corresponding Author, Zheng Wang, upon reasonable request.

Ethics Approval: Not applicable.

Conflicts of Interest: The authors declare no conflicts of interest.

References

1. Zhang Z, Xiao Y, Ma Z, Xiao M, Ding Z, Lei X, et al. 6G wireless networks: vision, requirements, architecture, and key technologies. IEEE Veh Technol Mag. 2019;14(3):28–41. doi:10.1109/MVT.2019.2921208. [Google Scholar] [CrossRef]

2. Liu J, Shi Y, Fadlullah ZM, Kato N. Space-air-ground integrated network: a survey. IEEE Commun Surv Tut. 2018;20(4):2714–41. doi:10.1109/comst.2018.2841996. [Google Scholar] [CrossRef]

3. Cai Y, Cheng P, Chen Z, Xiang W, Vucetic B, Li Y. Graphic deep reinforcement learning for dynamic resource allocation in space-air-ground integrated networks. IEEE J Sel Areas Commun. 2025;43(1):334–49. doi:10.1109/jsac.2024.3460086. [Google Scholar] [CrossRef]

4. Li H, Yu J, Cao L, Zhang Q, Song Z, Hou S. Multi-agent reinforcement learning based computation offloading and resource allocation for LEO satellite edge computing networks. Comput Commun. 2024;222(4):268–76. doi:10.2139/ssrn.4611047. [Google Scholar] [CrossRef]

5. Huang C, Chen G, Xiao P, Xiao Y, Han Z, Chambers JA. Joint offloading and resource allocation for hybrid cloud and edge computing in SAGINs: a decision assisted hybrid action space deep reinforcement learning approach. IEEE J Sel Areas Commun. 2024;42(5):1029–43. doi:10.1109/JSAC.2024.3365899. [Google Scholar] [CrossRef]

6. Liu L, Mao W, Li W, Duan J, Liu G, Guo B. Edge computing offloading strategy for space-air-ground integrated network based on game theory. Comput Netw. 2024;243(10):110331. doi:10.1016/j.comnet.2024.110331. [Google Scholar] [CrossRef]

7. Wachi A, Shen X, Sui Y. A survey of constraint formulations in safe reinforcement learning. In: Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI); 2024 Aug 3–9; Jeju, Republic of Korea. p. 8262–71. [Google Scholar]

8. Mao Y, Zhang J, Letaief KB. Dynamic computation offloading for mobile-edge computing with energy harvesting devices. IEEE J Sel Areas Commun. 2016;34(12):3590–3605. doi:10.1109/jsac.2016.2611964. [Google Scholar] [CrossRef]

9. Chen X. Decentralized computation offloading game for mobile cloud computing. IEEE Trans Parallel Distrib Syst. 2015;26(4):974–83. doi:10.1109/tpds.2014.2316834. [Google Scholar] [CrossRef]

10. Zhang J, Hu X, Ning Z, Ngai ECH, Zhou L, Wei J, et al. Energy-latency tradeoff for energy-aware offloading in mobile edge computing networks. IEEE Internet Things J. 2018;5(4):2633–45. doi:10.1109/jiot.2017.2786343. [Google Scholar] [CrossRef]

11. Geng L, Zhao H, Wang J, Kaushik A, Yuan SWK, Feng W. Deep reinforcement learning-based distributed computation offloading in vehicular edge computing networks. IEEE Internet Things J. 2023;10(14):12416–33. doi:10.1109/jiot.2023.3247013. [Google Scholar] [CrossRef]

12. Xiao Y, Ye Z, Wu M, Li H, Xiao M, Alouini MS, et al. Space-air-ground integrated wireless networks for 6G: basics, key technologies, and future trends. IEEE J Sel Areas Commun. 2024;42(12):3327–54. [Google Scholar]

13. Zhang J, Yang X, Chen X, Chen X, Yi X, Khalil I, et al. Energy-efficient UAV deployment and computation offloading in space-air-ground integrated networks. IEEE Trans Veh Technol. 2026;75(2):3081–98. doi:10.1109/tvt.2025.3601197. [Google Scholar] [CrossRef]

14. Zhang P, Li Y, Kumar N, Chen N, Hsu CH, Barnawi A. Distributed deep reinforcement learning assisted resource allocation algorithm for space-air-ground integrated networks. IEEE Trans Netw Serv Manag. 2023;20(3):3348–58. doi:10.1109/tnsm.2022.3232414. [Google Scholar] [CrossRef]

15. Liu Y, Jiang L, Qi Q, Xie K, Xie S. Online computation offloading for collaborative space/aerial-aided edge computing toward 6G system. IEEE Trans Veh Technol. 2024;73(2):2495–505. doi:10.1109/tvt.2023.3312676. [Google Scholar] [CrossRef]

16. Zhang X, Liu J, Zhang R, Huang Y, Tong J, Xin N, et al. Energy-efficient computation peer offloading in satellite edge computing networks. IEEE Trans Mob Comput. 2024;23(4):3077–91. doi:10.1109/tmc.2023.3269801. [Google Scholar] [CrossRef]

17. Bhattacherjee D, Aqeel W, Bozkurt IN, Aguirre A, Chandrasekaran B, Godfrey PB, et al. Gearing up for the 21st century space race. In: Proceedings of the 17th ACM Workshop on Hot Topics in Networks. New York, NY, USA: ACM; 2018. p. 113–9. [Google Scholar]

18. Mozaffari M, Saad W, Bennis M, Nam YH, Debbah M. A tutorial on UAVs for wireless networks: applications, challenges, and open problems. IEEE Commun Surv Tut. 2019;21(3):2334–60. [Google Scholar]

19. Li M, Cheng N, Gao J, Wang Y, Zhao L, Shen X. Energy-efficient UAV-assisted mobile edge computing: resource allocation and trajectory optimization. IEEE Trans Veh Technol. 2020;69(3):3424–38. [Google Scholar]

20. Yan M, Xiong R, Wang Y, Li C. Edge computing task offloading optimization for a UAV-assisted Internet of Vehicles via deep reinforcement learning. IEEE Trans Veh Technol. 2024;73(4):5647–58. doi:10.1109/tvt.2023.3331363. [Google Scholar] [CrossRef]

21. Mao Y, You C, Zhang J, Huang K, Letaief KB. A survey on mobile edge computing: the communication perspective. IEEE Commun Surv Tut. 2017;19(4):2322–58. doi:10.1109/comst.2017.2745201. [Google Scholar] [CrossRef]

22. Zhang Y, Hu J, Min G. Digital twin-driven intelligent task offloading for collaborative mobile edge computing. IEEE J Sel Areas Commun. 2023;41(10):3034–45. doi:10.1109/jsac.2023.3310058. [Google Scholar] [CrossRef]

23. Chen X, Jiao L, Li W, Fu X. Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans Netw. 2016;24(5):2795–808. doi:10.1109/tnet.2015.2487344. [Google Scholar] [CrossRef]

24. Xie M, Ye J, Zhang G, Ni X. Deep reinforcement learning-based computation offloading and distributed edge service caching for mobile edge computing. Comput Netw. 2024;250(9):110564. doi:10.1016/j.comnet.2024.110564. [Google Scholar] [CrossRef]

25. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, et al. Human-level control through deep reinforcement learning. Nature. 2015;518(7540):529–33. doi:10.1038/nature14236. [Google Scholar] [PubMed] [CrossRef]

26. Chen X, Zhang H, Wu C, Mao S, Ji Y, Bennis M. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet Things J. 2018;6(3):4005–18. doi:10.1109/jiot.2018.2876279. [Google Scholar] [CrossRef]

27. Zhu X, Zhang T, Zhang J, Zhao B, Zhang S, Wu C. Deep reinforcement learning-based edge computing offloading algorithm for software-defined IoT. Comput Netw. 2023;235:110006. doi:10.21203/rs.3.rs-2150294/v1. [Google Scholar] [CrossRef]

28. Ratnabala L, Peter R, Fedoseev A, Tsetserukou D. Hippo-mat: decentralized task allocation using GraphSAGE and multi-agent deep reinforcement learning. arXiv:2503.07662. 2025. [Google Scholar]

29. Meng K, Zhang S, Li R, Meng X, Deng Y, Wang C, et al. Multi-agent conditional diffusion model with mean field communication as wireless resource allocation planner. arXiv:2510.22969. 2025. [Google Scholar]

30. Jia K, Xia H, Zhang R, Sun Y, Wang K. Multi-agent DRL for edge computing: a real-time proportional compute offloading. Comput Netw. 2024;252:110665. [Google Scholar]

31. Shao Z, Yang H, Xiao L, Su W, Chen Y, Xiong Z. Deep reinforcement learning-based resource management for UAV-assisted mobile edge computing against jamming. IEEE Trans Mob Comput. 2024;23(12):13358–70. doi:10.1109/globecom54140.2023.10437090. [Google Scholar] [CrossRef]

32. Achiam J, Held D, Tamar A, Abbeel P. Constrained policy optimization. In: ICML’17: Proceedings of the 34th International Conference on Machine Learning; 2017 Aug 6–11; Sydney, NSW, Australia. p. 22–31. [Google Scholar]

33. Tessler C, Mankowitz DJ, Mannor S. Reward constrained policy optimization. arXiv:1805.11074. 2018. [Google Scholar]

34. Zhang Y, Vuong Q, Ross K. First order constrained optimization in policy space. Adv Neural Inf Process Syst. 2020;33:15338–49. doi:10.52202/075280-1703. [Google Scholar] [CrossRef]

35. Cao K, Chen M, Karnouskos S, Hu S. Reliability-aware personalized deployment of approximate computation IoT applications in serverless mobile edge computing. IEEE Trans Comput Aided Des Integr Circ Syst. 2025;44(2):430–43. doi:10.1109/tcad.2024.3437344. [Google Scholar] [CrossRef]

36. Li Z, Yu H, Fan G, Zhang J, Xu J. Energy-efficient reliability-aware offloading for delay-sensitive tasks in collaborative edge computing. Concurr Comput: Pract Exp. 2024;36(13):e8083. doi:10.1002/cpe.8083. [Google Scholar] [CrossRef]

37. Tang J, Nie J, Zhang Y, Xiong Z, Jiang W, Guizani M. Multi-UAV-assisted federated learning for energy-aware distributed edge training. IEEE Trans Netw Serv Manag. 2024;21(1):280–93. doi:10.1109/tnsm.2023.3298220. [Google Scholar] [CrossRef]

38. Qu Y, Zhang T, Feng Y, Xu T, Guo Z. Computation offloading and resource allocation for E2E tasks in satellite edge computing networks. Space: Sci Technol. 2024;4:0144. [Google Scholar]

39. Xue Z, Liu C, Liao C, Han G, Sheng Z. Joint service caching and computation offloading scheme based on deep reinforcement learning in vehicular edge computing systems. IEEE Trans Veh Technol. 2023;72(5):6709–22. doi:10.1109/tvt.2023.3234336. [Google Scholar] [CrossRef]

40. Wang Y, Sheng M, Wang X, Wang L, Li J. Mobile-edge computing: partial computation offloading using dynamic voltage scaling. IEEE Trans Commun. 2016;64(10):4268–82. [Google Scholar]

41. 3GPP. Study on channel model for frequencies from 0.5 to 100 GHz. 3GPP TR 38.901, Version 19.2.0; 2026 [cited 2026 May 10]. Available from: https://www.etsi.org/deliver/etsi_tr/138900_138999/138901/19.02.00_60/tr_138901v190200p.pdf. [Google Scholar]

42. ITU-R. Recommendation ITU-R P.618-14: propagation data and prediction methods required for the design of Earth-space telecommunication systems. International Telecommunication Union; 2023 [cited 2026 May 10]. Available from: https://www.itu.int/rec/R-REC-P.618-14-202308-I/en. [Google Scholar]

43. Atzori L, Iera A, Morabito G. The internet of things: a survey. Comput Netw. 2010;54(15):2787–805. doi:10.1016/j.comnet.2010.05.010. [Google Scholar] [CrossRef]

44. Khan LU, Yaqoob I, Tran NH, Kazmi SMA, Dang TN, Hong CS. Edge-computing-enabled smart cities: a comprehensive survey. IEEE Internet Things J. 2020;7(10):10200–232. [Google Scholar]

45. Abbas N, Zhang Y, Taherkordi A, Skeie T. Mobile edge computing: a survey. IEEE Internet Things J. 2017;5(1):450–65. doi:10.1109/jiot.2017.2750180. [Google Scholar] [CrossRef]