Open Access

ARTICLE

Real-Time Human Interaction Mimicry Teleoperation in Unitree G1 Edu Humanoid Robots Using the RGB Sensor

Yi Wen Tan¹, Jun Meng Woh¹, Ee Sin Yong¹, Wai Leong Pang¹, Hui Hwang Goh¹, Kah Yoong Chan², Ari Happonen^3,*

1 School of Engineering, Faculty of Innovation & Technology, Taylor’s University, Subang Jaya, Malaysia
2 Faculty of Artificial Intelligence and Engineering, Multimedia University, Cyberjaya, Malaysia
3 Software Engineering, School of Engineering Science, LUT University, Lappeenranta, Finland

* Corresponding Author: Ari Happonen. Email:

Computers, Materials & Continua 2026, 89(1), 58 https://doi.org/10.32604/cmc.2026.083418

Received 03 April 2026; Accepted 23 June 2026; Issue published 13 August 2026

Abstract

Along with the rapid advancement of Artificial Intelligence (AI), humanoid robots are foreseen to have great potential in the service industry, where human interaction is unavoidable. However, current systems face significant hurdles, including Field of View (FoV) problems, markerless real-time mimicry capabilities for humanoid’s fingers and arms. This study addresses these hurdles by developing an integrated hardware and software pipeline for the Unitree G1 Edu humanoid robot. A custom 3D-printed helmet and stabiliser interface were designed using FreeCAD and fabricated to house an external Orbbec Gemini 2 RGB-D sensor, optimising the FoV for frontal human-robot interaction by compensating for the robot’s height and its downward-facing onboard sensors. Both finger control and arm control use MediaPipe Holistic to extract the coordinates and perform kinematics calculation to obtain the angle commands for each joint, which are then passed through a first-order Exponential Moving Average (EMA) Filter for jitter avoidance. The safety mechanism was verified in MuJoCo simulator, followed by physical implementation on G1 humanoid. Under single-subject test in controlled environment, MediaPipe shows perception robustness of almost 100% tracking reliability for arm-level mimicry across various distances, lighting and occlusion conditions, but experiences serious performance degradation under severe backlighting. Low Root Mean Square Errors (RMSE) of 5.3°–5.6° for shoulder elevation and 8.7°–8.8° for elbow flexion, demonstrating high retargeting reproducibility. Evaluation of the jitter suppression stage indicates that the EMA filter successfully attenuated high-frequency pose estimation noise, yielding a 40.3%–63.4% jitter reduction in shoulder channels and a 34.2%–38.5% reduction in elbow channels while maintaining an optimal balance with tracking responsiveness. Deployment of the algorithm onto G1 Edu humanoid successfully verified the feasibility of the developed algorithm. Combining the algorithm with the customised helmet, this system provides a new framework for human-mimicking robotics development. Future work will focus on sensor fusion, multi-subject and dynamic environment test to better synchronise robot movement with real-world interaction scenario, which are vital for moving the technology from the lab to public service applications.

---

Keywords

---