Open Access

ARTICLE

Fusing Geometric and Temporal Deep Features for High-Precision Arabic Sign Language Recognition

Yazeed Alkhrijah^1,2, Shehzad Khalid³, Syed Muhammad Usman^4,*, Amina Jameel³, Danish Hamid⁵

1 King Salman Center for Disability Research (KSCDR), Riyadh, 11614, Saudi Arabia
2 Department of Electrical Engineering, Imam Mohammad ibn Saud Islamic University (IMSIU), Riyadh, 11623, Saudi Arabia
3 Department of Computer Engineering, Bahria University, Islamabad, 44000, Pakistan
4 Department of Computer Science, Bahria University, Islamabad, 44000, Pakistan
5 Department of Creative Technologies, Air University, Islamabad, 44000, Pakistan

* Corresponding Author: Syed Muhammad Usman. Email:

(This article belongs to the Special Issue: Machine Learning and Deep Learning-Based Pattern Recognition)

Computer Modeling in Engineering & Sciences 2025, 144(1), 1113-1141. https://doi.org/10.32604/cmes.2025.068726

Received 05 June 2025; Accepted 04 July 2025; Issue published 31 July 2025

Abstract

Arabic Sign Language (ArSL) recognition plays a vital role in enhancing the communication for the Deaf and Hard of Hearing (DHH) community. Researchers have proposed multiple methods for automated recognition of ArSL; however, these methods face multiple challenges that include high gesture variability, occlusions, limited signer diversity, and the scarcity of large annotated datasets. Existing methods, often relying solely on either skeletal data or video-based features, struggle with generalization and robustness, especially in dynamic and real-world conditions. This paper proposes a novel multimodal ensemble classification framework that integrates geometric features derived from 3D skeletal joint distances and angles with temporal features extracted from RGB videos using the Inflated 3D ConvNet (I3D). By fusing these complementary modalities at the feature level and applying a majority-voting ensemble of XGBoost, Random Forest, and Support Vector Machine classifiers, the framework robustly captures both spatial configurations and motion dynamics of sign gestures. Feature selection using the Pearson Correlation Coefficient further enhances efficiency by reducing redundancy. Extensive experiments on the ArabSign dataset, which includes RGB videos and corresponding skeletal data, demonstrate that the proposed approach significantly outperforms state-of-the-art methods, achieving an average F1-score of 97% using a majority-voting ensemble of XGBoost, Random Forest, and SVM classifiers, and improving recognition accuracy by more than 7% over previous best methods. This work not only advances the technical state-of-the-art in ArSL recognition but also provides a scalable, real-time solution for practical deployment in educational, social, and assistive communication technologies. Even though this study is about Arabic Sign Language, the framework proposed here can be extended to different sign languages, creating possibilities for potentially worldwide applicability in sign language recognition tasks.

---

Keywords

---