TY  - EJOU
AU  - Kamakaev, Daniil 
AU  - Mahbub, Khaled 

TI  - Artificially Intelligent Interviewer—A Multimodal Approach
T2  - Journal on Artificial Intelligence

PY  - 2026
VL  - 8
IS  - 1
SN  - 2579-003X

AB  - This paper presents an innovative system designed to automate the analysis of candidate interviews by integrating multiple analytical techniques into a single multimodal framework. This system combines text sentiment analysis, audio sentiment analysis, keyword extraction, and Mel-Frequency Cepstral Coefficients (MFCC) feature extraction to evaluate candidate performance holistically. This system employs text sentiment analysis using VADER and transformer-based sentiment features (probability-based outputs), audio sentiment analysis with an SVM model trained on both IEMOCAP and MELD datasets, keyword extraction via KeyBERT, and audio feature extraction including MFCCs, delta MFCCs, pitch, and energy to evaluate candidate performance holistically. A novel weighted scoring mechanism, incorporating personality traits such as Neuroticism, distinguishes this project from existing single-modality systems, offering a comprehensive candidate assessment. Results highlight its potential for Human Resources (HR) applications, with future improvements planned to enhance Negative sentiment detection through hybrid approaches and advanced emotion recognition techniques.
KW  - VADER (Valence Aware Dictionary and sEntiment Reasoner); MFCC (Mel-Frequency Cepstral Coefficients); KeyBERT; RoBERTa; openSMILE (Open-Source Media Interpretation by Large feature-space Extraction); MELD (Multimodal EmotionLines Dataset); CMU-MOSI (CMU Multimodal Opinion Sentiment Intensity); IEMOCAP (Interactive EMOtional dyadic motion CAPture database)

DO  - 10.32604/jai.2026.077823