Open Access

ARTICLE

Appearance Based Dynamic Hand Gesture Recognition Using 3D Separable Convolutional Neural Network

Muhammad Rizwan^1,*, Sana Ul Haq^1,*, Noor Gul^1,2, Muhammad Asif¹, Syed Muslim Shah³, Tariqullah Jan⁴, Naveed Ahmad⁵

1 Department of Electronics, University of Peshawar, Peshawar, 25120, Pakistan
2 Department of Electronics Engineering, Korea Polytechnic University, Siheung, Korea
3 Department of Electrical Engineering, Capital University of Science and Technology, Islamabad, 44000, Pakistan
4 Department of Electrical Engineering, University of Engineering and Technology, Peshawar, 25120, Pakistan
5 Department of Computer Science, Prince Sultan University, Riyadh, 11586, Saudi Arabia

* Corresponding Authors: Muhammad Rizwan. Email: ; Sana Ul Haq. Email:

Computers, Materials & Continua 2023, 76(1), 1213-1247. https://doi.org/10.32604/cmc.2023.038211

Received 02 December 2022; Accepted 28 March 2023; Issue published 08 June 2023

Abstract

Appearance-based dynamic Hand Gesture Recognition (HGR) remains a prominent area of research in Human-Computer Interaction (HCI). Numerous environmental and computational constraints limit its real-time deployment. In addition, the performance of a model decreases as the subject’s distance from the camera increases. This study proposes a 3D separable Convolutional Neural Network (CNN), considering the model’s computational complexity and recognition accuracy. The 20BN-Jester dataset was used to train the model for six gesture classes. After achieving the best offline recognition accuracy of 94.39%, the model was deployed in real-time while considering the subject’s attention, the instant of performing a gesture, and the subject’s distance from the camera. Despite being discussed in numerous research articles, the distance factor remains unresolved in real-time deployment, which leads to degraded recognition results. In the proposed approach, the distance calculation substantially improves the classification performance by reducing the impact of the subject’s distance from the camera. Additionally, the capability of feature extraction, degree of relevance, and statistical significance of the proposed model against other state-of-the-art models were validated using t-distributed Stochastic Neighbor Embedding (t-SNE), Mathew’s Correlation Coefficient (MCC), and the McNemar test, respectively. We observed that the proposed model exhibits state-of-the-art outcomes and a comparatively high significance level.

---

Keywords

---