Open Access iconOpen Access


Micro-Expression Recognition Based on Spatio-Temporal Feature Extraction of Key Regions

Wenqiu Zhu1,2, Yongsheng Li1,2, Qiang Liu1,2,*, Zhigao Zeng1,2

1 College of Computer Science, Hunan University of Technology, Zhuzhou, 412007, China
2 Intelligent Information Perception and Processing Technology, Hunan Province Key Laboratory, Zhuzhou, 412007, China

* Corresponding Author: Qiang Liu. Email: email

Computers, Materials & Continua 2023, 77(1), 1373-1392.


Aiming at the problems of short duration, low intensity, and difficult detection of micro-expressions (MEs), the global and local features of ME video frames are extracted by combining spatial feature extraction and temporal feature extraction. Based on traditional convolution neural network (CNN) and long short-term memory (LSTM), a recognition method combining global identification attention network (GIA), block identification attention network (BIA) and bi-directional long short-term memory (Bi-LSTM) is proposed. In the BIA, the ME video frame will be cropped, and the training will be carried out by cropping into 24 identification blocks (IBs), 10 IBs and uncropped IBs. To alleviate the overfitting problem in training, we first extract the basic features of the pre-processed sequence through the transfer learning layer, and then extract the global and local spatial features of the output data through the GIA layer and the BIA layer, respectively. In the BIA layer, the input data will be cropped into local feature vectors with attention weights to extract the local features of the ME frames; in the GIA layer, the global features of the ME frames will be extracted. Finally, after fusing the global and local feature vectors, the ME time-series information is extracted by Bi-LSTM. The experimental results show that using IBs can significantly improve the model's ability to extract subtle facial features, and the model works best when 10 IBs are used.


Cite This Article

APA Style
Zhu, W., Li, Y., Liu, Q., Zeng, Z. (2023). Micro-expression recognition based on spatio-temporal feature extraction of key regions. Computers, Materials & Continua, 77(1), 1373-1392.
Vancouver Style
Zhu W, Li Y, Liu Q, Zeng Z. Micro-expression recognition based on spatio-temporal feature extraction of key regions. Comput Mater Contin. 2023;77(1):1373-1392
IEEE Style
W. Zhu, Y. Li, Q. Liu, and Z. Zeng "Micro-Expression Recognition Based on Spatio-Temporal Feature Extraction of Key Regions," Comput. Mater. Contin., vol. 77, no. 1, pp. 1373-1392. 2023.

cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 304


  • 189


  • 0


Share Link