Open AccessOpen Access


Deep Facial Emotion Recognition Using Local Features Based on Facial Landmarks for Security System

Youngeun An, Jimin Lee, EunSang Bak*, Sungbum Pan*

IT Research Institute, Chosun University, Gwang-Ju, 61452, Korea

* Corresponding Authors: Eunsang Bak. Email: ; Sungbum Pan. Email:

(This article belongs to this Special Issue: Advances in Information Security Application)

Computers, Materials & Continua 2023, 76(2), 1817-1832.


Emotion recognition based on facial expressions is one of the most critical elements of human-machine interfaces. Most conventional methods for emotion recognition using facial expressions use the entire facial image to extract features and then recognize specific emotions through a pre-trained model. In contrast, this paper proposes a novel feature vector extraction method using the Euclidean distance between the landmarks changing their positions according to facial expressions, especially around the eyes, eyebrows, nose, and mouth. Then, we apply a new classifier using an ensemble network to increase emotion recognition accuracy. The emotion recognition performance was compared with the conventional algorithms using public databases. The results indicated that the proposed method achieved higher accuracy than the traditional based on facial expressions for emotion recognition. In particular, our experiments with the FER2013 database show that our proposed method is robust to lighting conditions and backgrounds, with an average of 25% higher performance than previous studies. Consequently, the proposed method is expected to recognize facial expressions, especially fear and anger, to help prevent severe accidents by detecting security-related or dangerous actions in advance.


Cite This Article

Y. An, J. Lee, E. Bak and S. Pan, "Deep facial emotion recognition using local features based on facial landmarks for security system," Computers, Materials & Continua, vol. 76, no.2, pp. 1817–1832, 2023.

This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 106


  • 72


  • 0


Share Link