Vol.27, No.1, 2021, pp.143-158, doi:10.32604/iasc.2021.012268
Research into Visual Servo Based Haptic Feedback Teleoperation
  • Tao Ni1, Lingtao Huang1,*, Huanfei Zheng2, Hongyan Zhang1
1 School of Mechanical and Aerospace Engineering, Jilin University, Changchun, 130000, China
2 Department of Mechanical Engineering, Clemson University, Clemson, 25633, USA
* Corresponding Author: Lingtao Huang. Email:
Received 23 June 2020; Accepted 10 August 2020; Issue published 07 January 2021
To overcome the problem that a teleoperation system loses robustness when the target moves outside the robot’s visual field or it is far away from the desired position, and to improve the operability and controllability of a master-slave teleoperation system, we present an image servo based haptic feedback (ISBHF) control method for teleoperating. The ISBHF control method involves extracting target feature points and constructing image servo based virtual force. First, the image characteristics of the environment and targets are identified and extracted by a 3D reconstruction method. A composite image Jacobian matrix is used to construct virtual guidance force based on the target’s image error. Then, the virtual force is provided for the operator through two joysticks which prevents the target image from drifting out of the visual field of the camera. We conducted experiments to confirm the validity and efficacy of the ISBHF control method. We found that human manipulation may cause the target image to deviate from the camera view, but the image servo based virtual force guides the operator to calibrate the target image back to the center view. The ISBHF control method can easily guide the robot to track the target and prevent image feature points deviating from the camera view. Teleoperation can be improved by combination of the robot visual sensory information and human haptic perception with the ISBHF control method. The ISBHF control method is favorable for the robot to avoid the blind operation of human and the collision with other obstacles.
Visual servo; composite image; Jacobian matrix; virtual force; haptic feedback
Cite This Article
T. Ni, L. Huang, H. Zheng and H. Zhang, "Research into visual servo based haptic feedback teleoperation," Intelligent Automation & Soft Computing, vol. 27, no.1, pp. 143–158, 2021.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.