A Web-Based Platform for Real-Time Speech Emotion Recognition using CNN

Damir Kabdualiyev, Askar Madiyev, Adil Rakhaliyev, Balgynbek Dikhan, Kassymzhan Gizhduan, Hashim Ali

2023

Full text Open access: Green Metadata (CSL-JSON)
DOI 10.1109/smartnets58706.2023.10215937
Data confidence 85 high
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

This pilot study presents a web-based real-time speech emotion recognition platform using a convolutional neural network algorithm. The study aims to develop a reliable tool for predicting emotions in speech with a user-friendly design to enable easy access and display of recognition results. The platform recognizes seven emotions (angry, disgust, fear, happy, neutral, sad, and surprise) and has two functionalities: static and real-time speech signals analysis. The static analysis allows users to upload pre-recorded audio files for analysis, while the real-time analysis provides continuous audio processing as it is being recorded. The study also focuses on developing a reliable model with minimal features to predict emotions while accurately identifying various emotions detected in speech. The algorithmic performance of the model was evaluated using publicly available datasets (RAVDESS, TESS, and SAVEE). It achieved an accuracy of 86.46% in static analysis using the selected spectral feature: i.e., MFCC. The performance of the real-time analysis was validated through a user study involving 20 participants. It achieved an accuracy of 65% in recognizing emotions in real-time due to possible known factors. An interesting finding was the discrepancy between how individuals perceived their emotions and those detected by the ML model. The accuracy of the ML model was higher in pre-recorded audio recognition and about the same in real-time recognition compared to previous works. The user-friendly design and CNN algorithm make it a promising solution to address challenges in emotion recognition and highlight the importance of further research in this field.