TY  - EJOU
AU  - Tan, Yi Wen 
AU  - Woh, Jun Meng 
AU  - Yong, Ee Sin 
AU  - Pang, Wai Leong 
AU  - Goh, Hui Hwang 
AU  - Chan, Kah Yoong 
AU  - Happonen, Ari 

TI  - Real-Time Human Interaction Mimicry Teleoperation in Unitree G1 Edu Humanoid Robots Using the RGB Sensor
T2  - Computers, Materials \& Continua

PY  - 
VL  - 
IS  - 
SN  - 1546-2226

AB  - Along with the rapid advancement of Artificial Intelligence (AI), humanoid robots are foreseen to have great potential in the service industry, where human interaction is unavoidable. However, current systems face significant hurdles, including Field of View (FoV) problems, markerless real-time mimicry capabilities for humanoid’s fingers and arms. This study addresses these hurdles by developing an integrated hardware and software pipeline for the Unitree G1 Edu humanoid robot. A custom 3D-printed helmet and stabiliser interface were designed using FreeCAD and fabricated to house an external Orbbec Gemini 2 RGB-D sensor, optimising the FoV for frontal human-robot interaction by compensating for the robot’s height and its downward-facing onboard sensors. Both finger control and arm control use MediaPipe Holistic to extract the coordinates and perform kinematics calculation to obtain the angle commands for each joint, which are then passed through a first-order Exponential Moving Average (EMA) Filter for jitter avoidance. The safety mechanism was verified in MuJoCo simulator, followed by physical implementation on G1 humanoid. Under single-subject test in controlled environment, MediaPipe shows perception robustness of almost 100% tracking reliability for arm-level mimicry across various distances, lighting and occlusion conditions, but experiences serious performance degradation under severe backlighting. Low Root Mean Square Errors (RMSE) of 5.3°–5.6° for shoulder elevation and 8.7°–8.8° for elbow flexion, demonstrating high retargeting reproducibility. Evaluation of the jitter suppression stage indicates that the EMA filter successfully attenuated high-frequency pose estimation noise, yielding a 40.3%–63.4% jitter reduction in shoulder channels and a 34.2%–38.5% reduction in elbow channels while maintaining an optimal balance with tracking responsiveness. Deployment of the algorithm onto G1 Edu humanoid successfully verified the feasibility of the developed algorithm. Combining the algorithm with the customised helmet, this system provides a new framework for human-mimicking robotics development. Future work will focus on sensor fusion, multi-subject and dynamic environment test to better synchronise robot movement with real-world interaction scenario, which are vital for moving the technology from the lab to public service applications.
KW  - Machine learning; humanoid; field of view; real-time processing; artificial intelligence; dynamic environment; industry 5.0; mimicry capability; RGB-D; human-robot interaction

DO  - 10.32604/cmc.2026.083418