Open Access iconOpen Access

ARTICLE

crossmark

Image-Based Lifelogging: User Emotion Perspective

Junghyun Bum1, Hyunseung Choo1, Joyce Jiyoung Whang2,*

1 College of Computing, Sungkyunkwan University, Suwon, 16417, Korea
2 School of Computing, KAIST, Daejeon, 34141, Korea

* Corresponding Author: Joyce Jiyoung Whang. Email: email

Computers, Materials & Continua 2021, 67(2), 1963-1977. https://doi.org/10.32604/cmc.2021.014931

Abstract

Lifelog is a digital record of an individual’s daily life. It collects, records, and archives a large amount of unstructured data; therefore, techniques are required to organize and summarize those data for easy retrieval. Lifelogging has been utilized for diverse applications including healthcare, self-tracking, and entertainment, among others. With regard to the image-based lifelogging, even though most users prefer to present photos with facial expressions that allow us to infer their emotions, there have been few studies on lifelogging techniques that focus upon users’ emotions. In this paper, we develop a system that extracts users’ own photos from their smartphones and configures their lifelogs with a focus on their emotions. We design an emotion classifier based on convolutional neural networks (CNN) to predict the users’ emotions. To train the model, we create a new dataset by collecting facial images from the CelebFaces Attributes (CelebA) dataset and labeling their facial emotion expressions, and by integrating parts of the Radboud Faces Database (RaFD). Our dataset consists of 4,715 high-resolution images. We propose Representative Emotional Data Extraction Scheme (REDES) to select representative photos based on inferring users’ emotions from their facial expressions. In addition, we develop a system that allows users to easily configure diaries for a special day and summaize their lifelogs. Our experimental results show that our method is able to effectively incorporate emotions into lifelog, allowing an enriched experience.

Keywords


Cite This Article

APA Style
Bum, J., Choo, H., Whang, J.J. (2021). Image-based lifelogging: user emotion perspective. Computers, Materials & Continua, 67(2), 1963-1977. https://doi.org/10.32604/cmc.2021.014931
Vancouver Style
Bum J, Choo H, Whang JJ. Image-based lifelogging: user emotion perspective. Comput Mater Contin. 2021;67(2):1963-1977 https://doi.org/10.32604/cmc.2021.014931
IEEE Style
J. Bum, H. Choo, and J.J. Whang "Image-Based Lifelogging: User Emotion Perspective," Comput. Mater. Contin., vol. 67, no. 2, pp. 1963-1977. 2021. https://doi.org/10.32604/cmc.2021.014931



cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1892

    View

  • 1179

    Download

  • 0

    Like

Share Link