Open Access

ARTICLE

Emotion Based Signal Enhancement Through Multisensory Integration Using Machine Learning

Muhammad Adnan Khan^1,2, Sagheer Abbas³, Ali Raza³, Faheem Khan⁴, T. Whangbo^4,*

1 Pattern Recognition and Machine Learning Lab, Department of Software, Gachon University, Seongnam, Gyeonggido, 13120, Korea
2 Riphah School of Computing & Innovation, Faculty of Computing, Riphah International University Lahore Campus, Lahore, 54000, Pakistan
3 School of Computer Science, National College of Business Administration and Economics, Lahore, 54000, Pakistan
4 Department of Computer Engineering, Gachon University, Seongnam, 13557, Korea

* Corresponding Author: T. Whangbo. Email:

Computers, Materials & Continua 2022, 71(3), 5911-5931. https://doi.org/10.32604/cmc.2022.023557

Received 12 September 2021; Accepted 13 October 2021; Issue published 14 January 2022

Abstract

Progress in understanding multisensory integration in human have suggested researchers that the integration may result into the enhancement or depression of incoming signals. It is evident based on different psychological and behavioral experiments that stimuli coming from different perceptual modalities at the same time or from the same place, the signal having more strength under the influence of emotions effects the response accordingly. Current research in multisensory integration has not studied the effect of emotions despite its significance and natural influence in multisensory enhancement or depression. Therefore, there is a need to integrate the emotional state of the agent with incoming stimuli for signal enhancement or depression. In this study, two different neural network-based learning algorithms have been employed to learn the impact of emotions on signal enhancement or depression. It was observed that the performance of a proposed system for multisensory integration increases when emotion features were present during enhancement or depression of multisensory signals.

---

Keywords

---