Open Access

ARTICLE

STR-CMFNet: A Visual-Tactile Spatio-Temporal Rectification and Cross-Modal Fusion Network for Slip Detection

Hao Chu, Xibin Xiao, Song Gao, Chao Zheng, Fei Wang^*

Faculty of Robot Science and Engineering, Northeastern University, Shenyang, China

* Corresponding Author: Fei Wang. Email:

Computers, Materials & Continua 2026, 89(1), 30 https://doi.org/10.32604/cmc.2026.083466

Received 04 April 2026; Accepted 16 June 2026; Issue published 13 August 2026

Abstract

In recent years, embodied intelligence has become an important research direction. Slip detection is a critical challenge in dexterous robotic manipulation, especially in manipulating deformable objects such as soft fruits. Visual–tactile fusion can provide rich sensory information, but it also introduces significant modality differences. We propose STR-CMFNet, a novel network framework for visual–tactile slip detection. The framework adopts a two-stage design. It consists of Spatio-Temporal Rectification (STR) and Cross-Modal Fusion (CMF). The STR applies temporal attention weights to recalibrate key-frame features. It also refines spatial feature maps. Based on the corrected features, the CMF module is further introduces. The global dependencies between visual and tactile features is modeled through a cross-attention mechanism to achieve effective cross-modal interactions and allows complementary information. Experiments are conducted on a public benchmark dataset and an extended fruit dataset. The results show that STR-CMFNet outperforms existing methods. In particular, while ViViT achieves a classification accuracy of 78.5%, the proposed method achieves 83.1% accuracy, showing a clear and consistent performance improvement. The ablation study further validates the effectiveness of each module. Overall, we provide a practical solution for stable robotic grasping of deformable objects, and demonstrate strong potential in complex interaction scenarios.

---

Keywords

---