A Learning-Driven Visual Servoing Framework for Latency Compensation in Image-Guided Teleoperation

Junmin Lyu¹, Feng Bao^2,*, Guangyu Xu³, Siyu Lu^4,*, Bo Yang⁵, Yuxin Liu⁵, Wenfeng Zheng⁵
1 School of Artificial Intelligence, Guangzhou Huashang University, Guangzhou, 511300, China
2 School of Biological and Environmental Engineering, Xi’an University, Xi’an, 710065, China
3 School of the Environment, The University of Queensland, St Lucia 2, Brisbane, 4072, Australia
4 Department of Geography, Texas A&M University, College Station, TX 77843, USA
5 School of Automation, University of Electronic Science and Technology of China, Chengdu, 611731, China
* Corresponding Author: Feng Bao. Email: email ; Siyu Lu. Email: email

Computer Modeling in Engineering & Sciences https://doi.org/10.32604/cmes.2025.075178

Received 27 October 2025; Accepted 10 December 2025; Published online 19 December 2025

Download PDF

Abstract

Robust teleoperation in image-guided interventions faces critical challenges from latency, deformation, and the quasi-periodic nature of physiological motion. This paper presents a fully integrated, latency-aware visual servoing system leveraging stereo vision, hand–eye calibration, and learning-based prediction for motion-compensated teleoperation. The system combines a calibrated binocular camera setup, dual robotic arms, and a predictive control loop incorporating Long Short-Term Memory (LSTM) and Temporal Convolutional Network (TCN) models. Through experiments using both in vivo and phantom datasets, we quantitatively assess the prediction accuracy and motion-compensation performance of both models. Results show that TCNs deliver more stable and precise tracking, especially on regular trajectories, while LSTMs exhibit robustness under quasi-periodic dynamics. By matching prediction horizons to system latency, the approach significantly reduces peak and steady-state tracking errors, demonstrating practical feasibility for deploying prediction-augmented servoing in teleoperated surgical.