Open Access

ARTICLE

Artificially Intelligent Interviewer—A Multimodal Approach

Daniil Kamakaev, Khaled Mahbub^*

Birmingham City University, 4 Cardigan Street, Birmingham, UK

* Corresponding Author: Khaled Mahbub. Email:

Journal on Artificial Intelligence 2026, 8, 183-202. https://doi.org/10.32604/jai.2026.077823

Received 17 December 2025; Accepted 26 February 2026; Issue published 15 April 2026

Abstract

This paper presents an innovative system designed to automate the analysis of candidate interviews by integrating multiple analytical techniques into a single multimodal framework. This system combines text sentiment analysis, audio sentiment analysis, keyword extraction, and Mel-Frequency Cepstral Coefficients (MFCC) feature extraction to evaluate candidate performance holistically. This system employs text sentiment analysis using VADER and transformer-based sentiment features (probability-based outputs), audio sentiment analysis with an SVM model trained on both IEMOCAP and MELD datasets, keyword extraction via KeyBERT, and audio feature extraction including MFCCs, delta MFCCs, pitch, and energy to evaluate candidate performance holistically. A novel weighted scoring mechanism, incorporating personality traits such as Neuroticism, distinguishes this project from existing single-modality systems, offering a comprehensive candidate assessment. Results highlight its potential for Human Resources (HR) applications, with future improvements planned to enhance Negative sentiment detection through hybrid approaches and advanced emotion recognition techniques.

---

Keywords

---