Open Access iconOpen Access



Research into Visual Servo Based Haptic Feedback Teleoperation

Tao Ni1, Lingtao Huang1,*, Huanfei Zheng2, Hongyan Zhang1

1 School of Mechanical and Aerospace Engineering, Jilin University, Changchun, 130000, China
2 Department of Mechanical Engineering, Clemson University, Clemson, 25633, USA

* Corresponding Author: Lingtao Huang. Email: email

Intelligent Automation & Soft Computing 2021, 27(1), 143-158.


To overcome the problem that a teleoperation system loses robustness when the target moves outside the robot’s visual field or it is far away from the desired position, and to improve the operability and controllability of a master-slave teleoperation system, we present an image servo based haptic feedback (ISBHF) control method for teleoperating. The ISBHF control method involves extracting target feature points and constructing image servo based virtual force. First, the image characteristics of the environment and targets are identified and extracted by a 3D reconstruction method. A composite image Jacobian matrix is used to construct virtual guidance force based on the target’s image error. Then, the virtual force is provided for the operator through two joysticks which prevents the target image from drifting out of the visual field of the camera. We conducted experiments to confirm the validity and efficacy of the ISBHF control method. We found that human manipulation may cause the target image to deviate from the camera view, but the image servo based virtual force guides the operator to calibrate the target image back to the center view. The ISBHF control method can easily guide the robot to track the target and prevent image feature points deviating from the camera view. Teleoperation can be improved by combination of the robot visual sensory information and human haptic perception with the ISBHF control method. The ISBHF control method is favorable for the robot to avoid the blind operation of human and the collision with other obstacles.


Cite This Article

APA Style
Ni, T., Huang, L., Zheng, H., Zhang, H. (2021). Research into visual servo based haptic feedback teleoperation. Intelligent Automation & Soft Computing, 27(1), 143-158.
Vancouver Style
Ni T, Huang L, Zheng H, Zhang H. Research into visual servo based haptic feedback teleoperation. Intell Automat Soft Comput . 2021;27(1):143-158
IEEE Style
T. Ni, L. Huang, H. Zheng, and H. Zhang "Research into Visual Servo Based Haptic Feedback Teleoperation," Intell. Automat. Soft Comput. , vol. 27, no. 1, pp. 143-158. 2021.

cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1900


  • 1051


  • 0


Related articles

Share Link