Chang Su, Liangliang Zhao^*, Dongbing Xiang

CMC-Computers, Materials & Continua, Vol.86, No.2, pp. 1-24, 2026, DOI:10.32604/cmc.2025.071319 - 09 December 2025

Abstract To address low learning efficiency and inadequate path safety in spraying robot navigation within complex obstacle-rich environments—with dense, dynamic, unpredictable obstacles challenging conventional methods—this paper proposes a hybrid algorithm integrating Q-learning and improved A*-Artificial Potential Field (A-APF). Centered on the Q-learning framework, the algorithm leverages safety-oriented guidance generated by A-APF and employs a dynamic coordination mechanism that adaptively balances exploration and exploitation. The proposed system comprises four core modules: (1) an environment modeling module that constructs grid-based obstacle maps; (2) an A-APF module that combines heuristic search from A* algorithm with repulsive force strategies from… More >

Wei Liu^1,*, Ruiyang Wang¹, Guangwei Liu²

CMC-Computers, Materials & Continua, Vol.86, No.2, pp. 1-23, 2026, DOI:10.32604/cmc.2025.070328 - 09 December 2025

Abstract Q-learning is a classical reinforcement learning method with broad applicability. It can respond effectively to environmental changes and provide flexible strategies, making it suitable for solving robot path-planning problems. However, Q-learning faces challenges in search and update efficiency. To address these issues, we propose an improved Q-learning (IQL) algorithm. We use an enhanced Ant Colony Optimization (ACO) algorithm to optimize Q-table initialization. We also introduce the UCH mechanism to refine the reward function and overcome the exploration dilemma. The IQL algorithm is extensively tested in three grid environments of different scales. The results validate the… More >

Vi Hoai Nam¹, Chu Thi Minh Hue², Dang Van Anh^1,*

CMC-Computers, Materials & Continua, Vol.86, No.1, pp. 1-15, 2026, DOI:10.32604/cmc.2025.070605 - 10 November 2025

Abstract Unmanned Aerial Vehicles (UAVs) have become integral components in smart city infrastructures, supporting applications such as emergency response, surveillance, and data collection. However, the high mobility and dynamic topology of Flying Ad Hoc Networks (FANETs) present significant challenges for maintaining reliable, low-latency communication. Conventional geographic routing protocols often struggle in situations where link quality varies and mobility patterns are unpredictable. To overcome these limitations, this paper proposes an improved routing protocol based on reinforcement learning. This new approach integrates Q-learning with mechanisms that are both link-aware and mobility-aware. The proposed method optimizes the selection of… More >

Xiaoyu Wen^1,2, Haohao Liu^1,2, Xinyu Zhang^1,2, Haoqi Wang^1,2, Yuyan Zhang^1,2, Guoyong Ye^1,2, Hongwen Xing³, Siren Liu³, Hao Li^1,2,*

CMC-Computers, Materials & Continua, Vol.86, No.1, pp. 1-27, 2026, DOI:10.32604/cmc.2025.069492 - 10 November 2025

Abstract Aircraft assembly is characterized by stringent precedence constraints, limited resource availability, spatial restrictions, and a high degree of manual intervention. These factors lead to considerable variability in operator workloads and significantly increase the complexity of scheduling. To address this challenge, this study investigates the Aircraft Pulsating Assembly Line Scheduling Problem (APALSP) under skilled operator allocation, with the objective of minimizing assembly completion time. A mathematical model considering skilled operator allocation is developed, and a Q-Learning improved Particle Swarm Optimization algorithm (QLPSO) is proposed. In the algorithm design, a reverse scheduling strategy is adopted to effectively… More >

Sheng Bi^1,2,*, Jiayan Wang¹, Dong Su¹, Hui Lu¹, Yu Zhang¹

Energy Engineering, Vol.122, No.10, pp. 4135-4151, 2025, DOI:10.32604/ee.2025.066735 - 30 September 2025

Abstract By comparing price plans offered by several retail energy firms, end users with smart meters and controllers may optimize their energy use cost portfolios, due to the growth of deregulated retail power markets. To help smart grid end-users decrease power payment and usage unhappiness, this article suggests a decision system based on reinforcement learning to aid with electricity price plan selection. An enhanced state-based Markov decision process (MDP) without transition probabilities simulates the decision issue. A Kernel approximate-integrated batch Q-learning approach is used to tackle the given issue. Several adjustments to the sampling and data… More >

Nimisha Rajput^1,#, Amit Kumar¹, Raghavendra Pal^1,#, Nishu Gupta^2,*, Mikko Uitto², Jukka Mäkelä²

CMES-Computer Modeling in Engineering & Sciences, Vol.143, No.3, pp. 3839-3859, 2025, DOI:10.32604/cmes.2025.065903 - 30 June 2025

Abstract Wireless Sensor Networks (WSNs) play a critical role in automated border surveillance systems, where continuous monitoring is essential. However, limited energy resources in sensor nodes lead to frequent network failures and reduced coverage over time. To address this issue, this paper presents an innovative energy-efficient protocol based on deep Q-learning (DQN), specifically developed to prolong the operational lifespan of WSNs used in border surveillance. By harnessing the adaptive power of DQN, the proposed protocol dynamically adjusts node activity and communication patterns. This approach ensures optimal energy usage while maintaining high coverage, connectivity, and data accuracy. More >

Song Gao, Shixin Liu^*

CMC-Computers, Materials & Continua, Vol.83, No.3, pp. 5623-5641, 2025, DOI:10.32604/cmc.2025.058334 - 19 May 2025

Abstract With the development of economic globalization, distributed manufacturing is becoming more and more prevalent. Recently, integrated scheduling of distributed production and assembly has captured much concern. This research studies a distributed flexible job shop scheduling problem with assembly operations. Firstly, a mixed integer programming model is formulated to minimize the maximum completion time. Secondly, a Q-learning-assisted co-evolutionary algorithm is presented to solve the model: (1) Multiple populations are developed to seek required decisions simultaneously; (2) An encoding and decoding method based on problem features is applied to represent individuals; (3) A hybrid approach of heuristic… More >

Naeem Ali Askar^1,*, Adib Habbal^1,*, Hassen Hamouda², Abdullah Mohammad Alnajim³, Sheroz Khan⁴

CMC-Computers, Materials & Continua, Vol.81, No.3, pp. 4625-4658, 2024, DOI:10.32604/cmc.2024.058607 - 19 December 2024

Abstract Named Data Networking (NDN) has emerged as a promising communication paradigm, emphasizing content-centric access rather than location-based access. This model offers several advantages for Internet of Healthcare Things (IoHT) environments, including efficient content distribution, built-in security, and natural support for mobility and scalability. However, existing NDN-based IoHT systems face inefficiencies in their forwarding strategy, where identical Interest packets are forwarded across multiple nodes, causing broadcast storms, increased collisions, higher energy consumption, and delays. These issues negatively impact healthcare system performance, particularly for individuals with disabilities and chronic diseases requiring continuous monitoring. To address these challenges,… More >

Jehad Ali^1,*, Mohammed J. F. Alenazi²

CMC-Computers, Materials & Continua, Vol.81, No.3, pp. 4015-4032, 2024, DOI:10.32604/cmc.2024.058480 - 19 December 2024

Abstract The controller is a main component in the Software-Defined Networking (SDN) framework, which plays a significant role in enabling programmability and orchestration for 5G and next-generation networks. In SDN, frequent communication occurs between network switches and the controller, which manages and directs traffic flows. If the controller is not strategically placed within the network, this communication can experience increased delays, negatively affecting network performance. Specifically, an improperly placed controller can lead to higher end-to-end (E2E) delay, as switches must traverse more hops or encounter greater propagation delays when communicating with the controller. This paper introduces… More >

Qianyao Zhu¹, Kaizhou Gao^1,*, Wuze Huang¹, Zhenfang Ma¹, Adam Slowik²

CMC-Computers, Materials & Continua, Vol.80, No.3, pp. 3573-3589, 2024, DOI:10.32604/cmc.2024.055244 - 12 September 2024

Abstract The flow shop scheduling problem is important for the manufacturing industry. Effective flow shop scheduling can bring great benefits to the industry. However, there are few types of research on Distributed Hybrid Flow Shop Problems (DHFSP) by learning assisted meta-heuristics. This work addresses a DHFSP with minimizing the maximum completion time (Makespan). First, a mathematical model is developed for the concerned DHFSP. Second, four Q-learning-assisted meta-heuristics, e.g., genetic algorithm (GA), artificial bee colony algorithm (ABC), particle swarm optimization (PSO), and differential evolution (DE), are proposed. According to the nature of DHFSP, six local search operations… More >