Cooperative Task Offloading in Mobile Edge Computing via an Improved MASAC Framework

1  Introduction

Intelligent mobile applications such as environmental perception, object recognition, path planning, and cooperative control require timely computation, yet mobile terminals are limited by onboard computing capacity and battery energy. Cloud-only execution provides abundant resources but introduces long transmission paths and response delay. MEC mitigates this limitation by placing computation near terminals and enabling task offloading between local and edge resources.

Cooperative offloading remains difficult in dynamic MEC networks because terminal mobility changes wireless links, stochastic arrivals create time-varying workloads, and multiple terminals compete for shared edge resources. This paper therefore studies cooperative task offloading in dynamic MEC networks, jointly considering node association, task offloading ratio, and transmit-power control under communication, computation, and queueing constraints. The resulting problem is modeled as a multi-agent continuous decision-making problem under CTDE, and an improved MASAC-based method is developed to address bounded action representation, exploration adaptation, and training stability. The main contributions are summarized as follows.

(1)   An MEC system model is established for cooperative task offloading in dynamic mobile-edge network scenarios. The model captures terminal mobility, time-varying wireless links, spatially heterogeneous task arrivals, and shared edge-resource competition. Based on this model, the task-offloading problem is formulated to reduce latency and energy consumption while suppressing queue accumulation.

(2)   Under the CTDE framework, node association, task offloading, and transmit power control are unified into a multi-agent joint decision-making problem. Considering local observation constraints and joint state transition characteristics, the problem is further modeled as a decentralized partially observable Markov decision process (Dec-POMDP).

(3)   To adapt MASAC to the bounded and relaxed hybrid action structure of cooperative MEC offloading, an improved MASAC-based method, named ABDMASAC, is developed. Different from standard MASAC, the proposed method jointly incorporates Beta-policy-based bounded action modeling, adaptive entropy regulation, and delayed actor updates into the CTDE framework. This design enables the agents to learn node-association preferences, offloading ratios, and transmit-power decisions under dynamic link states, stochastic task arrivals, and shared edge-resource competition. These modules are not claimed to independently improve all performance metrics. Instead, they are designed as an integrated learning-stabilization mechanism, where the Beta policy provides bounded action representation, adaptive entropy tuning regulates exploration, and delayed actor updates improve late-stage training stability.

(4)   Comparative experiments are conducted under a unified training budget and a consistent evaluation protocol. The results show that ABDMASAC achieves a better overall trade-off than the selected MASAC-backbone and on-policy MARL baselines in terms of system reward, end-to-end latency, and terminal-side energy consumption.

The remainder of this paper is organized as follows. Section 2 gives the system model and problem formulation. Section 3 presents ABDMASAC. Section 4 reports experiments and ablations, and Section 5 concludes the paper.

1.1 Related Work

1.1.1 Optimization-Based MEC Task Offloading

Optimization-based MEC studies have modeled offloading through power-delay tradeoffs, resource allocation, trajectory planning, dense-user interaction, task heterogeneity, and multi-objective scheduling. Representative works include Chen et al. [1], Mao et al. [2], Wu et al. [3], Yang et al. [4], Li et al. [5], Yao et al. [6], Gao et al. [7], Tong et al. [8], Han et al. [9], Misha et al. [10], and Yao and Chang [11]. Surveys by Mao et al. [12], Feng et al. [13], Kar et al. [14], Dong et al. [15], and Wang et al. [16] show that MEC offloading has evolved toward joint latency-energy-resource optimization, but many models still rely on simplified dynamics and coordination assumptions.

1.1.2 DRL-Based MEC Task Offloading

DRL has been used to improve adaptability when MEC states and constraints are difficult to optimize analytically. Surveys by Luo and Dai [17], Hortelano et al. [18], Zabihi et al. [19], Peng et al. [20], and Nabi et al. [21] summarize this trend. Representative studies include Tang and Wong [22], Mirza et al. [23], Liu et al. [24], Liu et al. [25], and Xie et al. [26]. More recently, Mustafa et al. [27] developed a Dueling-DQN-based method for partial computation offloading and resource allocation in MEC, further demonstrating the effectiveness of value-based DRL in jointly optimizing task partitioning and resource-control decisions. However, single-agent DRL is less suitable when multiple terminals jointly affect node load, delay, energy, and queue evolution.

1.1.3 MARL-Based Cooperative Offloading

MARL-based MEC offloading has considered communication-assisted coordination, UAV trajectory coupling, prioritized replay, and hybrid action spaces, as shown by Tan et al. [28], Ju et al. [29], Shi et al. [30], and Wang et al. [31]. The evolutionary-game-and-MATD3-based joint node-selection and task-offloading method of Yao et al. [32] further addresses UAV-assisted MEC with hybrid discrete-continuous action spaces. Chen et al. [33], Zuo et al. [34], Li et al. [35], and Bo and Zhao [36] extend MARL offloading to LEO, SAGIN, V2X, and vehicle-edge settings. These studies support cooperative learning, but bounded hybrid actions and stable CTDE training remain challenging under mobility, stochastic arrivals, and edge-resource contention.

1.1.4 Research Gap and Motivation

Existing studies usually emphasize discrete association, continuous resource control, or simplified coordination. The joint treatment of relaxed service-node association, bounded task partitioning and power control, time-varying links, stochastic arrivals, and shared edge-resource competition remains insufficient, motivating the proposed ABDMASAC framework under CTDE.

2  System Model and Problem Formulation

This section defines the dynamic MEC model and the cooperative offloading problem. Robot terminals in Fig. 1 are only an example; the formulation uses the general term “terminal”.

Figure 1: A representative mobile-terminal MEC scenario for cooperative task offloading.

2.1 System Model

Consider the heterogeneous MEC system in Fig. 1, with multiple mobile terminals, one MBS, and K SBSs. The MBS and SBSs are edge nodes, and each task can be processed locally, by an SBS, or by the MBS.

Let the terminal set be defined as 𝒩={1,2,…,N}, where N denotes the number of mobile terminals. Let K denote the number of SBSs and J=K+1 denote the number of edge nodes, including one MBS and K SBSs. The service-option set is defined as 𝒮={0,1,…,J}, where 0 denotes local execution, 1 denotes the MBS, and 2,…,J denote the candidate SBSs. The edge-node set is defined as ℬ=𝒮∖{0}.

The system evolves in region ℛ⊂R2 over slots 𝒯={0,1,…,T−1} of length Δt. Each terminal selects a service option, offloading ratio, and transmit power from local observations, while edge resources are shared according to node load.

Let the two-dimensional positions of terminal n and service nodes be defined in Eq. (1):

pn(t)=[xn(t),yn(t)]T,n∈𝒩, t∈𝒯,q1=[x1mbs,y1mbs]T,qj=[xjsbs,yjsbs]T,j∈{2,…,J}.(1)

Terminal mobility is modeled by the Gaussian random walk in Eq. (2):

pn(t+1)=Πℛ(pn(t)+Δpn(t)),(2)

where Πℛ(⋅) keeps the terminal inside ℛ, and Δpn(t)∼N(0,σm2I).

The selected service option is j¯n(t)∈𝒮, where 0, 1, and 2,…,J denote local execution, the MBS, and SBSs, respectively. Positions, selections, and arrivals jointly determine node load.

2.2 Communication Model

Let cn,j(t)∈{0,1} denote the NLoS/LoS state of link (n,j). Temporal blockage correlation is described by a two-state Markov chain Pr(cn,j(t+1)∣cn,j(t)).

Let the Euclidean distance between terminal n and edge node j∈ℬ at slot t and the corresponding channel gain be defined by Eq. (3):

dn,j(t)=‖pn(t)−qj‖2,hn,j(t)=h(dn,j(t),cn,j(t)).(3)

Here, ‖⋅‖2 is the Euclidean norm, and h(⋅) depends on distance and link state.

Let un(t)∈[0,1] denote the normalized transmit-power control variable of terminal n at slot t. The corresponding actual transmit power is defined as Pn(t)=un(t)Pmax, where Pmax denotes the maximum transmit power.

An effective orthogonal uplink access model is used: concurrent uploads occupy orthogonal resource blocks or an effective allocated bandwidth, so co-channel inter-user interference is not explicitly modeled. Thus, B in Eq. (4) is the effective uplink bandwidth.

The uplink transmission rate from terminal n to edge node j∈ℬ at slot t is given in Eq. (4):

Rn,j(t)=Blog2(1+xn,j(t)Pn(t)hn,j(t)σ2),(4)

where B denotes the effective allocated uplink bandwidth under the orthogonal-access assumption, xn,j(t) indicates whether edge node j is selected for task transmission, and σ2 denotes the noise power.

Accordingly, if the amount of offloaded task data of terminal n at slot t is Dnoff(t), the uplink transmission delay and the corresponding transmission energy consumption to edge node j∈ℬ are defined in Eq. (5):

Tn,jtx(t)=Dnoff(t)Rn,j(t),En,jtx(t)=Pn(t)Tn,jtx(t).(5)

2.3 Computation Model

Assume that the task generated by terminal n at slot t is represented by the triplet in Eq. (6):

Ωn(t)=(Dn(t),Cn(t),τnmax(t)).(6)

Here, Dn(t), Cn(t), and τnmax(t) denote data size, CPU cycles per bit, and latency tolerance, respectively.

Let Qn(t) be the terminal-side task queue. Its evolution is

Qn(t+1)=max{Qn(t)−Sn(t),0}+An(t),(7)

where An(t) and Sn(t) denote newly arrived and completed tasks.

With offloading ratio ρn(t)∈[0,1], local and offloaded data are

Dnloc(t)=(1−ρn(t))Dn(t),Dnoff(t)=ρn(t)Dn(t).(8)

For local frequency fnloc(t), local delay and energy are

Tnloc(t)=Dnloc(t)Cn(t)fnloc(t),Enloc(t)=κ(fnloc(t))2Dnloc(t)Cn(t),(9)

where κ is the effective capacitance coefficient.

For offloaded data, let Fj be the capacity of node j and 𝒩j(t)={n∈𝒩∣j¯n(t)=j}. The edge-processing delay is Tn,jedge(t)=Dnoff(t)Cn(t)fn,j(t). Under equal-share CPU allocation, fn,j(t)=Fj|𝒩j(t)| for n∈𝒩j(t).

This equal-share rule captures edge-resource contention. The model does not introduce a persistent server-side queue or explicit edge waiting time; backlog is tracked at the terminal side through Qn(t).

2.4 Problem Formulation

At slot t, terminal n controls association xn,j(t), offloading ratio ρn(t)∈[0,1], and normalized power un(t)∈[0,1]. The edge CPU share fn,j(t) is induced by node selection and load, not treated as an independent action.

The mixed decisions are relaxed into the continuous action vector

an(t)=(gn(t),ρn(t),un(t)).(10)

The node-association gating vector is defined in Eq. (11):

gn(t)=[gn,j(t)]j∈𝒮.(11)

Here, gn,j(t) is the association preference for option j. The executable option is obtained by

j^n(t)=arg⁡maxj∈𝒮gn,j(t).(12)

Accordingly, the binary association indicator is defined in Eq. (13):

xn,j(t)={1,j=j^n(t),0,otherwise,j∈𝒮.(13)

The gating vector is a continuous relaxation of discrete association. The argmax projection in Eq. (12) is used only during environment interaction and is not treated as differentiable. The policy is optimized in the relaxed continuous space, while the projected hard association is used to compute delay, energy, and queue evolution; hence the method is a relaxation-based hybrid-action policy learner.

Accordingly, the total end-to-end latency of terminal n at slot t is defined in Eq. (14):

Tn(t)=max(Tnloc(t),∑j∈ℬxn,j(t)(Tn,jtx(t)+Tn,jedge(t))),(14)

and the corresponding terminal-side energy consumption is defined in Eq. (15):

En(t)=Enloc(t)+∑j∈ℬxn,j(t)En,jtx(t).(15)

The per-slot cost combines latency, energy, queue length, and overload:

cn(t)=wdd¯n(t)+wee¯n(t)+wqq¯n(t)+woo¯n(t),(16)

where wd, we, wq, and wo denote the weights associated with latency, energy consumption, queue length, and overload penalty, respectively.

The normalized terms are

d¯n(t)=Tn(t)dref,e¯n(t)=En(t)eref,q¯n(t)=Qn(t)Qref,o¯n(t)=max(Qn(t)−Qth,0)Qref,(17)

where dref, eref, and Qref denote the reference scales for latency, energy consumption, and queue length, respectively, and Qth denotes the queue-overload threshold.

At the system level, the average cost at slot t is defined in Eq. (18):

C¯(t)=1N∑n=1Ncn(t).(18)

Accordingly, the long-term cooperative offloading objective can be formulated as Eq. (19):

minE[∑t=0T−1γtC¯(t)].(19)

where γ∈(0,1) is the discount factor.

The resulting problem is

P1:min{xn,j(t),ρn(t),un(t)} E[∑t=0T−1γtC¯(t)]s.t.xn,j(t)∈{0,1},∀n∈𝒩, j∈𝒮, t∈𝒯,∑j∈𝒮xn,j(t)=1,∀n∈𝒩, t∈𝒯,0≤ρn(t)≤1,0≤un(t)≤1,∀n∈𝒩, t∈𝒯,Qn(t+1)=max{Qn(t)−Sn(t),0}+An(t),fn,j(t)=Fj|𝒩j(t)|,n∈𝒩j(t), j∈ℬ.(20)

The constraints specify binary one-option association, bounded continuous controls, queue evolution, and equal-share CPU allocation.

Because P1 couples binary association, bounded continuous control, stochastic transitions, and multiple agents, it is solved as a Dec-POMDP under CTDE.

The shared reward is the negative average cost:

r(t)=−C¯(t).(21)

Reward clipping is used for critic stability:

r~(t)=clip⁡(r(t),−10,0).(22)

3  Cooperative Offloading Optimization Method Based on Improved MASAC

Following CTDE [37], P1 is formulated as a Dec-POMDP and solved by the proposed ABDMASAC algorithm.

3.1 Dec-POMDP Formulation

Each terminal is an agent, and the agent set is

𝒩={1,2,…,N}.(23)

The global state s(t)∈X contains terminal positions, link states, queues, and node loads. Agent n observes on(t)∈On, including its position, task state, reachable-link information, and local resources. The joint observation is

o(t)=(o1(t),o2(t),…,oN(t)).(24)

The local action an(t)∈An follows Eq. (10): the gating vector represents relaxed node association, ρn(t) is the offloading ratio, and un(t) is normalized transmit power. The discrete service option is obtained by Eq. (12). The joint action is

a(t)=(a1(t),a2(t),…,aN(t)).(25)

The transition probability is

P(s(t+1)∣s(t),a(t)),(26)

which is driven by mobility, link switching, task arrivals, and shared edge-resource competition. The shared reward follows Eq. (21); maximizing its discounted return is equivalent to minimizing the cost in P1.

3.2 Improved Learning Framework Based on MASAC

ABDMASAC builds on SAC [38] and CTDE [37], and improves MASAC through bounded action modeling, adaptive entropy tuning, and delayed policy updates.

3.2.1 CTDE Learning Framework

Under CTDE, each terminal executes its actor using only local observations, while centralized critics use global information and joint actions for training. Fig. 2 shows the framework.

Figure 2: Overall framework of the proposed ABDMASAC under centralized training and decentralized execution. The replay buffer stores the relaxed continuous joint action atrel, while the hard service-option selection is obtained by argmax projection only during environment interaction.

Let the actor-network parameters of terminal n be denoted by θn. The actor takes the local observation otn as input and outputs the continuous action atn. Twin critics Qϕnj(st,at), j∈{1,2}, use the global state and relaxed joint action; target networks and Huber loss improve stability [38,39]. The critic loss is

L(ϕnj)=E(st,at,rt,st+1)∼D[ℋδ(Qϕnj(st,at)−yt)],j∈{1,2},(27)

where D denotes the replay buffer, ℋδ(⋅) denotes the Huber loss, and the target value yt is defined in Eq. (28):

The replay buffer stores the relaxed continuous action vector, including gating, offloading ratio, and normalized power. Argmax-based hard association is used only by the environment, whereas the centralized critic uses the global state and relaxed joint action for value gradients.

yt=rt+γ(mink∈{1,2}Qϕnk′(st+1,at+1)−αlog⁡πθn(at+1n∣ot+1n))(28)

3.2.2 Beta Bounded Policy

Because gating variables, offloading ratio, and normalized power are bounded, ABDMASAC follows Chou et al. [40] and models each continuous action dimension with a Beta distribution.

The actor outputs (αtn,βtn)=fθn(otn), and samples

atn∼Beta⁡(αtn,βtn).(29)

In the implementation, the actor network outputs two unconstrained vectors, which are transformed into positive Beta concentration parameters through the softplus function. To avoid numerical instability near degenerate Beta distributions, the concentration parameters are clipped to [10−3,100]. During training, stochastic actions are sampled using the reparameterized sampling function of the Beta distribution, which provides a pathwise gradient estimator for the continuous relaxed action. During deterministic evaluation, the mean of the Beta distribution is used. The log-probability of the sampled bounded action is used in the entropy-regularized actor objective.

For x∈(0,1), the Beta density is

π(x;α,β)=Γ(α+β)Γ(α)Γ(β)xα−1(1−x)β−1,x∈(0,1),(30)

where Γ(⋅) is the Gamma function. At execution, gating is projected by Eq. (12), while ρn(t) and un(t) remain continuous controls.

The entropy-regularized actor objective is

J(θn)=E(st,at)∼D[αlog⁡πθn(atn∣otn)−Qϕn(st,at)](31)

3.2.3 Adaptive Entropy Tuning

ABDMASAC uses adaptive temperature α [38] to regulate exploration across training stages.

The temperature objective is

J(α)=Eatn∼πθn[−α(log⁡πθn(atn∣otn)+H¯)],(32)

where H¯ is the target entropy. Minimizing J(α) adjusts exploration according to the gap between current and target entropy.

3.2.4 Delayed Policy Update

To reduce actor oscillation caused by transient critic errors, ABDMASAC adopts delayed policy updates inspired by Fujimoto et al. [39].

The actor is updated only every d critic updates:

θn←θn−ηθ∇θnJ(θn),if tmod⁡d=0,(33)

here, d is the delay interval and ηθ is the actor learning rate. Target critics use soft updates ϕnj′←τϕnj+(1−τ)ϕnj′,j∈{1,2}.

3.3 Network Update and Training Procedure

During interaction, each actor samples a bounded relaxed action from its Beta policy; the gating part is projected to a service option only for environment execution. The environment returns r(t) and s(t+1), and r~(t)=clip⁡(r(t),−10,0) is stored.

The interaction sample is stored in the replay buffer as (s(t),o(t),arel(t),r~(t),s(t+1),o(t+1),done(t)), where arel(t) denotes the relaxed continuous joint action before the argmax projection, and done(t) denotes the terminal flag.

Unless specified, at in learning objectives denotes the relaxed continuous joint action.

Mini-batches update the twin critics using soft Bellman targets and Huber loss. The learned value estimates the long-term cooperative objective containing latency, energy, and queue terms.

Actors and entropy coefficient α are updated according to the delay interval and target-entropy objective.

Algorithm 1 summarizes the complete training procedure of ABDMASAC under the CTDE framework.

4  Simulation Results and Analysis

This section compares ABDMASAC with MASAC and MAPPO in small- and large-scale MEC scenarios and reports an ablation study.

4.1 Simulation Settings

The testbed contains one MBS and multiple SBSs. The small-scale scenario uses N=5 terminals and K=3 SBSs, while the large-scale scenario uses N=20 and K=4. Terminals follow Gaussian mobility, links follow LoS/NLoS transitions, and hotspot arrivals plus equal-share CPU allocation create controlled resource contention. Parameters are listed in Table 1; richer mobility, interference, bandwidth, CPU scheduling, and edge-server queueing are left for future work.

Two representative MARL baselines are selected in this study for a controlled comparison. MASAC is used as the direct backbone baseline because the proposed method is developed by improving the MASAC learning framework. This comparison helps evaluate whether the proposed bounded action modeling, adaptive entropy tuning, and delayed actor update can improve the original MASAC backbone under the same state representation, action parameterization, reward definition, training budget, and evaluation protocol. MAPPO is adopted as another representative on-policy MARL baseline based on a different policy-optimization paradigm.

It should be noted that the current comparison is not intended to serve as an exhaustive benchmark of all continuous-action or hybrid-action MARL algorithms for MEC. Recent hybrid-action MARL methods usually rely on different action decompositions, discrete-continuous coupling mechanisms, state definitions, and resource-allocation assumptions. Directly adapting these methods to the present relaxed-action CTDE environment would require additional redesign and implementation choices, which may introduce uncontrolled factors beyond the scope of this revision. Therefore, the experiments focus on a controlled comparison with the direct MASAC backbone and a representative on-policy MARL baseline. Broader comparisons with hybrid-action and continuous-action MARL methods under a unified MEC environment will be treated as an important direction for future work. Each method is trained for 50,000 steps, evaluated every 100 steps, and averaged over three seeds; Table 2 lists common settings.

Table 3 summarizes implementation details. SAC-based methods use decentralized actors and centralized twin critics under CTDE; ABDMASAC uses a Beta policy, whereas MASAC uses a tanh-squashed Gaussian policy. Other settings are kept identical unless specified.

Reward weights and target entropy are fixed across methods. The metrics are average return, end-to-end latency, energy consumption, and, for ablation, queue length.

4.2 Performance Comparison

4.2.1 Results in the Small-Scale Scenario

Fig. 3 shows that ABDMASAC obtains the best overall trade-off in the small-scale scenario. Its final return is about −1.85, better than MASAC (−2.75) and MAPPO (−2.40), and its latency is about 0.16 s, lower than MASAC (0.23 s) and MAPPO (0.20 s). MAPPO has the lowest energy, about 0.0112 vs. 0.0128 for ABDMASAC, but this comes with worse return and latency.

Figure 3: Performance comparison in the small-scale scenario: (a) average return, (b) average end-to-end latency, and (c) average energy consumption. The light curves denote the raw evaluation trajectories, while the dark curves denote the smoothed trends.

4.2.2 Results in the Large-Scale Scenario

Fig. 4 shows a clearer advantage at larger scale. ABDMASAC reaches about −1.02 return, 0.082 s latency, and 0.0050 energy, compared with MASAC (−1.24, 0.100 s, 0.0056) and MAPPO (−1.25, 0.101 s, 0.0074). Thus, ABDMASAC improves return while reducing both latency and energy consumption under stronger multi-agent competition.

Figure 4: Performance comparison in the large-scale scenario: (a) average return, (b) average end-to-end latency, and (c) average energy consumption. The light curves denote the raw evaluation trajectories, while the dark curves denote the smoothed trends.

4.3 Ablation Study

4.3.1 Ablation Settings

A reduced environment is used for ablation while retaining the main load and resource-competition pattern. Four variants are tested: Baseline (standard MASAC), Beta-only, Beta-adaptive, and Full (ABDMASAC). Table 4 lists their settings; means and standard deviations are reported over multiple seeds.

4.3.2 Ablation Results and Discussion

Figs. 5 and 6 show that Baseline obtains about −3.28 return, while Beta-only decreases to about −3.73, with latency increasing to 0.321 s and queue length to 0.67×10−4. The Beta policy mainly provides a bounded action representation, but using it alone may worsen learning because the exploration strength and update dynamics are not properly adjusted. Beta-adaptive gives the best return (−2.13), latency (0.183 s), and queue length (0.22×10−4), with energy about 0.0090. Full further lowers energy to 0.0088, but its return, latency, and queue length are about −2.50, 0.231 s, and 0.49×10−4, respectively. Thus, adaptive entropy tuning provides the clearest gain, while delayed updates adjust the final trade-off; the modules are complementary rather than independently monotonic.

Figure 5: Training-stage metric trajectories of different ablation variants under the Ablation-XS setting: (a) reward, (b) energy, (c) latency, and (d) queue length.

Figure 6: Statistical comparison over the last 20 evaluation points in the late training stage.

5  Conclusion

This paper investigates cooperative task offloading in dynamic MEC networks under CTDE and proposes ABDMASAC, which integrates Beta-policy-based bounded action modeling, adaptive entropy regulation, and delayed policy updates. Under the considered simulation settings, ABDMASAC achieves a better overall trade-off than the selected MASAC-backbone and on-policy MARL baselines. The ablation results show that the modules are complementary but not independently monotonic: the Beta policy alone may degrade performance when exploration and update dynamics are not properly adjusted, whereas the full method benefits from coordinated integration. The current comparison is not an exhaustive benchmark of all hybrid-action or continuous-action MARL methods. Future work will consider broader MARL comparisons, heterogeneous tasks, dynamic bandwidth allocation, communication overhead, explicit edge-server queueing, and security- and privacy-aware offloading.

Acknowledgement: Not applicable.

Funding Statement: This work was supported by the Youth Talent Project of the Scientific Research Program of the Yunnan Key Research and Development Program under Grant 202503AP140020, the Major Science and Technology Special Project of Yunnan Province under Grant 202502AD080003, the Hubei Provincial Department of Education under Grant Q20241809, and the Doctoral Scientific Research Foundation of Hubei University of Automotive Technology under Grant BK202404.

Author Contributions: Zheng Yao and Changjun Deng: conceptualization. Zheng Yao, Jie Liu and Changjun Deng: data curation; investigation. Zheng Yao and Jie Liu: writing—original draft. Zheng Yao, Jie Liu, Changjun Deng, and Wang Lin: writing—review & editing. All authors reviewed and approved the final version of the manuscript.

Availability of Data and Materials: The data supporting the findings of this study are available from the corresponding author upon reasonable request.

Ethics Approval: Not applicable.

Conflicts of Interest: Changjun Deng and Wang Lin are affiliated with Yunnan Hanzhe Technology Co., Ltd. through the Yunnan International Joint Laboratory of Agricultural Remote Sensing and Digital Technology. This affiliation may be perceived as a potential conflict of interest. The authors declare that this affiliation did not influence the study design, data analysis, interpretation of results, manuscript preparation, or decision to publish. The remaining authors declare no conflicts of interest.

References

1. Chen X, Jiao L, Li W, Fu X. Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans Netw. 2016;24(5):2795–808. doi:10.1109/tnet.2015.2487344. [Google Scholar] [CrossRef]

2. Mao Y, Zhang J, Song SH, Letaief KB. Power-delay tradeoff in multi-user mobile-edge computing systems. In: Proceedings of the 2016 IEEE Global Communications Conference (GLOBECOM); 2016 Dec 4–8; Washington, DC, USA. p. 1–6. [Google Scholar]

3. Wu Y, Qian LP, Ni K, Zhang C, Shen XS. Delay-minimization nonorthogonal multiple access enabled multi-user mobile edge computation offloading. IEEE J Sel Top Signal Process. 2019;13(3):392–407. doi:10.1109/jstsp.2019.2893057. [Google Scholar] [CrossRef]

4. Yang W, Liu Z, Liu X, Ma Y. Deep reinforcement learning-based low-latency task offloading for mobile-edge computing networks. Appl Soft Comput. 2024;166(1):112164. doi:10.1016/j.asoc.2024.112164. [Google Scholar] [CrossRef]

5. Li B, Liu W, Xie W, Li X. Energy-efficient task offloading and trajectory planning in UAV-enabled mobile edge computing networks. Comput Netw. 2023;234(1):109940. doi:10.1016/j.comnet.2023.109940. [Google Scholar] [CrossRef]

6. Yao Z, Zhu Q, Zhang Y, Huang H, Luo M. Minimizing long-term energy consumption in RIS-assisted AAV-enabled MEC network. IEEE Internet Things J. 2025;12(12):20942–58. doi:10.1109/jiot.2025.3545252. [Google Scholar] [CrossRef]

7. Gao H, Li W, Banez RA, Han Z, Poor HV. Mean field evolutionary dynamics in dense-user multi-access edge computing systems. IEEE Trans Wirel Commun. 2020;19(12):7825–35. doi:10.1109/twc.2020.3016695. [Google Scholar] [CrossRef]

8. Tong Z, Wang J, Mei J, Li K. Multi-type task offloading for wireless internet of things by federated deep reinforcement learning. Future Gener Comput Syst. 2023;145(11):536–49. doi:10.1016/j.future.2023.04.004. [Google Scholar] [CrossRef]

9. Han C, Chai Z, Li Y. Distributed task offloading in edge computing: a multi-objective adaptive deep reinforcement learning algorithm. Eng Appl Artif Intell. 2025;162:112653. [Google Scholar]

10. Misha T, Sun L, Chai ZY. Multi-objective multi-workflow task offloading based on evolutionary optimization. J King Saud Univ Comput Inf Sci. 2025;37(7):189. doi:10.1007/s44443-025-00073-8. [Google Scholar] [CrossRef]

11. Yao Z, Chang P. High-dimensional multi-objective computation offloading for MEC in serial isomerism tasks via flexible optimization framework. Comput Mater Contin. 2026;86(1):1–18. doi:10.32604/cmc.2025.068248. [Google Scholar] [CrossRef]

12. Mao Y, You C, Zhang J, Huang K, Letaief KB. A survey on mobile edge computing: the communication perspective. IEEE Commun Surv Tutor. 2017;19(4):2322–58. doi:10.1109/comst.2017.2745201. [Google Scholar] [CrossRef]

13. Feng C, Han P, Zhang X, Yang B, Liu Y, Guo L. Computation offloading in mobile edge computing networks: a survey. J Netw Comput Appl. 2022;202(1):103366. doi:10.1016/j.jnca.2022.103366. [Google Scholar] [CrossRef]

14. Kar B, Yahya W, Lin YR, Ali A. Offloading using traditional optimization and machine learning in federated cloud-edge-fog systems: a survey. IEEE Commun Surv Tutor. 2023;25(2):1199–226. doi:10.1109/comst.2023.3239579. [Google Scholar] [CrossRef]

15. Dong S, Tang J, Abbas K, Hou R, Kamruzzaman J, Rutkowski L, et al. Task offloading strategies for mobile edge computing: a survey. Comput Netw. 2024;254(6):110791. doi:10.1016/j.comnet.2024.110791. [Google Scholar] [CrossRef]

16. Wang D, Bin Abu Bakar K, Isyaku B, Eisa TAE, Abdelmaboud A. A comprehensive review on internet of things task offloading in multi-access edge computing. Heliyon. 2024;10(9):e29916. doi:10.1016/j.heliyon.2024.e29916. [Google Scholar] [PubMed] [CrossRef]

17. Luo Z, Dai X. Reinforcement learning-based computation offloading in edge computing: principles, methods, challenges. Alex Eng J. 2024;108(6):89–107. doi:10.1016/j.aej.2024.07.049. [Google Scholar] [CrossRef]

18. Hortelano D, de Miguel I, Duran Barroso RJ, Aguado JC, Merayo N, Ruiz L, et al. A comprehensive survey on reinforcement-learning-based computation offloading techniques in edge computing systems. J Netw Comput Appl. 2023;216(3):103669. doi:10.1016/j.jnca.2023.103669. [Google Scholar] [CrossRef]

19. Zabihi Z, Eftekhari Moghadam AM, Rezvani MH. Reinforcement learning methods for computation offloading: a systematic review. ACM Comput Surv. 2024;56(1):1–41. doi:10.1145/3603703. [Google Scholar] [CrossRef]

20. Peng P, Lin W, Wu W, Zhang H, Peng S, Wu Q, et al. A survey on computation offloading in edge systems: from the perspective of deep reinforcement learning approaches. Comput Sci Rev. 2024;53:100656. [Google Scholar]

21. Nabi A, Baidya T, Moh S. Comprehensive survey on reinforcement-learning-based task offloading techniques in aerial edge computing. Internet Things. 2024;28(3):101342. doi:10.1016/j.iot.2024.101342. [Google Scholar] [CrossRef]

22. Tang M, Wong VWS. Deep reinforcement learning for task offloading in mobile edge computing systems. IEEE Trans Mob Comput. 2022;21(6):1985–97. doi:10.1109/tmc.2020.3036871. [Google Scholar] [CrossRef]

23. Mirza MA, Yu J, Raza S, Krichen M, Ahmed M, Khan WU, et al. DRL-assisted delay optimized task offloading in automotive-industry 5.0 based VECNs. J King Saud Univ Comput Inf Sci. 2023;35(6):101512. doi:10.1016/j.jksuci.2023.02.013. [Google Scholar] [CrossRef]

24. Liu J, Mi Y, Zhang X, Li X. Task graph offloading via deep reinforcement learning in mobile edge computing. Future Gener Comput Syst. 2024;158(1):545–55. doi:10.1016/j.future.2024.04.034. [Google Scholar] [CrossRef]

25. Liu X, Chai ZY, Li YL, Cheng YY, Zeng Y. Multi-objective deep reinforcement learning for computation offloading in UAV-assisted multi-access edge computing. Inf Sci. 2023;642(1):119154. doi:10.1016/j.ins.2023.119154. [Google Scholar] [CrossRef]

26. Xie M, Ye J, Zhang G, Ni X. Deep reinforcement learning-based computation offloading and distributed edge service caching for mobile edge computing. Comput Netw. 2024;250(2):110564. doi:10.2139/ssrn.4725151. [Google Scholar] [CrossRef]

27. Mustafa E, Shuja J, Rehman F, Namoun A, Ali M, Alourani A. Deep reinforcement learning with dueling DQN for partial computation offloading and resource allocation in mobile edge computing. IEEE Access. 2025;13:94319–35. doi:10.1109/access.2025.3573929. [Google Scholar] [CrossRef]

28. Tan S, Chen B, Liu D, Zhang J, Hanzo L. Communication-assisted multi-agent reinforcement learning improves task-offloading in UAV-aided edge-computing networks. IEEE Wirel Commun Lett. 2023;12(12):2233–7. doi:10.1109/lwc.2023.3316794. [Google Scholar] [CrossRef]

29. Ju T, Li L, Liu S, Zhang Y. A multi-UAV assisted task offloading and path optimization for mobile edge computing via multi-agent deep reinforcement learning. J Netw Comput Appl. 2024;229(6):103919. doi:10.1016/j.jnca.2024.103919. [Google Scholar] [CrossRef]

30. Shi H, Tian Y, Li H, Huang J, Shi L, Zhou Y. Task offloading and trajectory scheduling for UAV-enabled MEC networks: an MADRL algorithm with prioritized experience replay. Ad Hoc Netw. 2024;154(1):103371. doi:10.1109/lwc.2021.3122957. [Google Scholar] [CrossRef]

31. Wang J, Zhang M, Yin Q, Yin L, Peng Y. Multi-agent reinforcement learning for task offloading with hybrid decision space in multi-access edge computing. Ad Hoc Netw. 2025;166(11):103671. doi:10.1016/j.adhoc.2024.103671. [Google Scholar] [CrossRef]

32. Yao Z, Chang P, Khalil FK, Duan C. Joint node selection and task offloading via evolutionary game and MATD3 in UAV-assisted MEC networks. J King Saud Univ Comput Inf Sci. 2025;37(8):220. doi:10.1007/s44443-025-00248-3. [Google Scholar] [CrossRef]

33. Chen J, Zhong J, Wu Z, Tian D, Chen Y. Priority-aware task offloading for LEO satellite edge computing network: a multi-agent deep reinforcement learning-based approach. J King Saud Univ Comput Inf Sci. 2025;37(7):168. doi:10.1007/s44443-025-00160-w. [Google Scholar] [CrossRef]

34. Zuo P, Miao C, Fu C, Wang X, Liu X, Liu B. SMAPPO: a security-aware multi-agent reinforcement learning framework for secure computation offloading in SAGIN. J King Saud Univ Comput Inf Sci. 2025;37:336. [Google Scholar]

35. Li J, Li J, Shi Y, Lian H, Wu H. A multi-agent deep reinforcement learning algorithm for task offloading in future 6G V2X network. IEICE Trans Inf Syst. 2025;E108-D(7):697–708. doi:10.1587/transinf.2024iip0005. [Google Scholar] [CrossRef]

36. Bo J, Zhao X. Vehicle edge computing task offloading strategy based on multi-agent deep reinforcement learning. J Grid Comput. 2025;23(2):13. doi:10.1007/s10723-025-09800-x. [Google Scholar] [CrossRef]

37. Lowe R, Wu Y, Tamar A, Harb J, Abbeel P, Mordatch I. Multi-agent actor-critic for mixed cooperative-competitive environments. In: Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017); 2017 Dec 4–9; Long Beach, CA, USA. [Google Scholar]

38. Haarnoja T, Zhou A, Abbeel P, Levine S. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: Proceedings of the 35th International Conference on Machine Learning (ICML); 2018 Jul 10–15; Stockholm, Sweden. [Google Scholar]

39. Fujimoto S, van Hoof H, Meger D. Addressing function approximation error in actor-critic methods. In: Proceedings of the 35th International Conference on Machine Learning (ICML); 2018 Jul 10–15; Stockholm, Sweden. p. 1587–96. [Google Scholar]

40. Chou PW, Maturana D, Scherer S. Improving stochastic policy gradients in continuous control with deep reinforcement learning using the beta distribution. In: Proceedings of the 34th International Conference on Machine Learning (ICML); 2017 Aug 6–11; Sydney, Australia. p. 834–43. [Google Scholar]