Deep Learning-Driven Decision Fusion: Spatio-Spectrogram Features for Inner Speech Recognition From Electroencephalogram Signals

2026

Full text Open access: Hybrid Metadata (CSL-JSON)
DOI 10.1109/thms.2026.3690175
Data confidence 85 high
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Inner speech recognition using electroencephalogram (EEG) signals shows strong potential for developing assistive communication technologies. Existing methods often process spatial and temporal features separately, lack interpretability, and are usually tested on a single dataset, limiting their generalization. This study proposes a dual-branch deep learning framework that combines spatial features extracted through common spatial patterns (CSPs) with spectral-temporal features derived from multitaper spectrograms, using convolutional and long short-term memory networks. The model was evaluated on two public datasets, achieving classification accuracies of 89.99% and 92.47% in subject-dependent experiments. Subject-independent evaluation using leave-one-subject-out cross-validation yielded reduced accuracies of 26.20% and 20.47%, reflecting intersubject variability. Interpretability analyses using saliency maps, gradient-weighted class activation mapping, and feature contribution ratios highlighted physiologically meaningful patterns related to model decisions. The proposed method demonstrates strong performance and interpretability for subject-dependent inner speech recognition; while future work will focus on increasing data diversity and improving subject-independent generalization. This study contributes to the development of reliable and explainable EEG-based inner speech decoding for communication applications.