Using BlazePose on Spatial Temporal Graph Convolutional Networks for Action Recognition

Motasem Alsawadi; El-Sayed El-kenawy; Miguel Rio

doi:10.32604/cmc.2023.032499

Table of Content

Open Access

ARTICLE

Using BlazePose on Spatial Temporal Graph Convolutional Networks for Action Recognition

Motasem S. Alsawadi^1,*, El-Sayed M. El-kenawy^2,3, Miguel Rio¹
1 Electronic and Electrical Engineering Department, University College London, London, WC1E 7JE, England
2 Department of Communications and Electronics, Delta Higher Institute of Engineering and Technology, Mansoura, 35111, Egypt
3 Faculty of Artificial Intelligence, Delta University for Science and Technology, Mansoura, 35712, Egypt
* Corresponding Author: Motasem S. Alsawadi. Email:

Computers, Materials & Continua 2023, 74(1), 19-36. https://doi.org/10.32604/cmc.2023.032499

Received 20 May 2022; Accepted 21 June 2022; Issue published 22 September 2022

Abstract

The ever-growing available visual data (i.e., uploaded videos and pictures by internet users) has attracted the research community's attention in the computer vision field. Therefore, finding efficient solutions to extract knowledge from these sources is imperative. Recently, the BlazePose system has been released for skeleton extraction from images oriented to mobile devices. With this skeleton graph representation in place, a Spatial-Temporal Graph Convolutional Network can be implemented to predict the action. We hypothesize that just by changing the skeleton input data for a different set of joints that offers more information about the action of interest, it is possible to increase the performance of the Spatial-Temporal Graph Convolutional Network for HAR tasks. Hence, in this study, we present the first implementation of the BlazePose skeleton topology upon this architecture for action recognition. Moreover, we propose the Enhanced-BlazePose topology that can achieve better results than its predecessor. Additionally, we propose different skeleton detection thresholds that can improve the accuracy performance even further. We reached a top-1 accuracy performance of 40.1% on the Kinetics dataset. For the NTU-RGB+D dataset, we achieved 87.59% and 92.1% accuracy for Cross-Subject and Cross-View evaluation criteria, respectively.

Keywords

Action recognition; BlazePose; graph neural network; OpenPose; skeleton; spatial temporal graph convolution network

Cite This Article

M. S. Alsawadi, E. M. El-kenawy and M. Rio, "Using blazepose on spatial temporal graph convolutional networks for action recognition," Computers, Materials & Continua, vol. 74, no.1, pp. 19–36, 2023.

This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.