Thai-Viet Dang^1,*, Dinh-Manh-Cuong Tran¹, Nhu-Nghia Bui¹, Phan Xuan Tan^2,*

CMC-Computers, Materials & Continua, Vol.85, No.2, pp. 2651-2680, 2025, DOI:10.32604/cmc.2025.067500 - 23 September 2025

Abstract Precise and robust three-dimensional object detection (3DOD) presents a promising opportunity in the field of mobile robot (MR) navigation. Monocular 3DOD techniques typically involve extending existing two-dimensional object detection (2DOD) frameworks to predict the three-dimensional bounding box (3DBB) of objects captured in 2D RGB images. However, these methods often require multiple images, making them less feasible for various real-time scenarios. To address these challenges, the emergence of agile convolutional neural networks (CNNs) capable of inferring depth from a single image opens a new avenue for investigation. The paper proposes a novel ELDE-Net network designed to… More >

Abubakar Elsafi^*

CMES-Computer Modeling in Engineering & Sciences, Vol.144, No.2, pp. 1749-1766, 2025, DOI:10.32604/cmes.2025.066449 - 31 August 2025

Abstract Domain randomization is a widely adopted technique in deep reinforcement learning (DRL) to improve agent generalization by exposing policies to diverse environmental conditions. This paper investigates the impact of different reset strategies, normal, non-randomized, and randomized, on agent performance using the Deep Deterministic Policy Gradient (DDPG) and Twin Delayed DDPG (TD3) algorithms within the CarRacing-v2 environment. Two experimental setups were conducted: an extended training regime with DDPG for 1000 steps per episode across 1000 episodes, and a fast execution setup comparing DDPG and TD3 for 30 episodes with 50 steps per episode under constrained computational… More >

Mingoo Kwon, Kyeongmin Kim, Minseok Song^*

CMC-Computers, Materials & Continua, Vol.85, No.1, pp. 519-537, 2025, DOI:10.32604/cmc.2025.066754 - 29 August 2025

Abstract Efficient edge caching is essential for maximizing utility in video streaming systems, especially under constraints such as limited storage capacity and dynamically fluctuating content popularity. Utility, defined as the benefit obtained per unit of cache bandwidth usage, degrades when static or greedy caching strategies fail to adapt to changing demand patterns. To address this, we propose a deep reinforcement learning (DRL)-based caching framework built upon the proximal policy optimization (PPO) algorithm. Our approach formulates edge caching as a sequential decision-making problem and introduces a reward model that balances cache hit performance and utility by prioritizing More >

Yi-Hao Tu, Yi-Wei Ma^*

CMC-Computers, Materials & Continua, Vol.84, No.3, pp. 4563-4582, 2025, DOI:10.32604/cmc.2025.066899 - 30 July 2025

Abstract This study introduces the Smart Exponential-Threshold-Linear with Double Deep Q-learning Network (SETL-DDQN) and an extended Gumbel distribution method, designed to optimize the Contention Window (CW) in IEEE 802.11 networks. Unlike conventional Deep Reinforcement Learning (DRL)-based approaches for CW size adjustment, which often suffer from overestimation bias and limited exploration diversity, leading to suboptimal throughput and collision performance. Our framework integrates the Gumbel distribution and extreme value theory to systematically enhance action selection under varying network conditions. First, SETL adopts a DDQN architecture (SETL-DDQN) to improve Q-value estimation accuracy and enhance training stability. Second, we incorporate a… More >

Venkata Satya Suresh kumar Kondeti¹, Raghavendra Kulkarni¹, Binu Sudhakaran Pillai², Surendran Rajendran^3,*

CMC-Computers, Materials & Continua, Vol.84, No.3, pp. 4973-4995, 2025, DOI:10.32604/cmc.2025.066428 - 30 July 2025

Abstract Next-generation 6G networks seek to provide ultra-reliable and low-latency communications, necessitating network designs that are intelligent and adaptable. Network slicing has developed as an effective option for resource separation and service-level differentiation inside virtualized infrastructures. Nonetheless, sustaining elevated Quality of Service (QoS) in dynamic, resource-limited systems poses significant hurdles. This study introduces an innovative packet-based proactive end-to-end (ETE) resource management system that facilitates network slicing with improved resilience and proactivity. To get around the drawbacks of conventional reactive systems, we develop a cost-efficient slice provisioning architecture that takes into account limits on radio, processing, and… More >

Yu-Hsien Lin^*, Po-Cheng Chuang, Joyce Yi-Tzu Huang

CMC-Computers, Materials & Continua, Vol.84, No.3, pp. 4907-4948, 2025, DOI:10.32604/cmc.2025.065995 - 30 July 2025

Abstract This study proposes an automatic control system for Autonomous Underwater Vehicle (AUV) docking, utilizing a digital twin (DT) environment based on the HoloOcean platform, which integrates six-degree-of-freedom (6-DOF) motion equations and hydrodynamic coefficients to create a realistic simulation. Although conventional model-based and visual servoing approaches often struggle in dynamic underwater environments due to limited adaptability and extensive parameter tuning requirements, deep reinforcement learning (DRL) offers a promising alternative. In the positioning stage, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is employed for synchronized depth and heading control, which offers stable training, reduced overestimation… More >

Qing Zhu¹, Denghui Guo¹, Rui Ruan¹, Zhidong Chai¹, Chaoqun Wang², Zhiwen Guan^2,*

Energy Engineering, Vol.122, No.8, pp. 3133-3154, 2025, DOI:10.32604/ee.2025.063165 - 24 July 2025

Abstract This study presents an emergency control method for sub-synchronous oscillations in wind power grid-connected systems based on transfer learning, addressing the issue of insufficient generalization ability of traditional methods in complex real-world scenarios. By combining deep reinforcement learning with a transfer learning framework, cross-scenario knowledge transfer is achieved, significantly enhancing the adaptability of the control strategy. First, a sub-synchronous oscillation emergency control model for the wind power grid integration system is constructed under fixed scenarios based on deep reinforcement learning. A reward evaluation system based on the active power oscillation pattern of the system is… More >

Wenlong Ke^1,2,*, Zilong Li¹, Peiyu Chen¹, Benfeng Chen¹, Jinglin Lv¹, Qiang Wang², Ziyi Jia², Shigen Shen¹

CMC-Computers, Materials & Continua, Vol.84, No.2, pp. 3305-3319, 2025, DOI:10.32604/cmc.2025.065665 - 03 July 2025

Abstract Zero Trust Network (ZTN) enhances network security through strict authentication and access control. However, in the ZTN, optimizing flow control to improve the quality of service is still facing challenges. Software Defined Network (SDN) provides solutions through centralized control and dynamic resource allocation, but the existing scheduling methods based on Deep Reinforcement Learning (DRL) are insufficient in terms of convergence speed and dynamic optimization capability. To solve these problems, this paper proposes DRL-AMIR, which is an efficient flow scheduling method for software defined ZTN. This method constructs a flow scheduling optimization model that comprehensively considers… More >

Qian Wang¹, Ya Zhou^1,2,*

CMC-Computers, Materials & Continua, Vol.84, No.2, pp. 3835-3856, 2025, DOI:10.32604/cmc.2025.065465 - 03 July 2025

Abstract Edge computing has transformed smart grids by lowering latency, reducing network congestion, and enabling real-time decision-making. Nevertheless, devising an optimal task-offloading strategy remains challenging, as it must jointly minimise energy consumption and response time under fluctuating workloads and volatile network conditions. We cast the offloading problem as a Markov Decision Process (MDP) and solve it with Deep Reinforcement Learning (DRL). Specifically, we present a three-tier architecture—end devices, edge nodes, and a cloud server—and enhance Proximal Policy Optimization (PPO) to learn adaptive, energy-aware policies. A Convolutional Neural Network (CNN) extracts high-level features from system states, enabling More >

Chenxi Lyu¹, Chen Dong¹, Qiancheng Xiong¹, Yuzhong Chen¹, Qian Weng^1,*, Zhenyi Chen²

CMC-Computers, Materials & Continua, Vol.84, No.2, pp. 3371-3391, 2025, DOI:10.32604/cmc.2025.065153 - 03 July 2025

Abstract The rapid advancement of Industry 4.0 has revolutionized manufacturing, shifting production from centralized control to decentralized, intelligent systems. Smart factories are now expected to achieve high adaptability and resource efficiency, particularly in mass customization scenarios where production schedules must accommodate dynamic and personalized demands. To address the challenges of dynamic task allocation, uncertainty, and real-time decision-making, this paper proposes Pathfinder, a deep reinforcement learning-based scheduling framework. Pathfinder models scheduling data through three key matrices: execution time (the time required for a job to complete), completion time (the actual time at which a job is finished),… More >