Leveraging Multihead Self‐Attention Mechanism in Convolutional Neural Networks for Enhanced Vibration‐Based Prediction of Transformer Fault

Fatemeh Ghanadian, Amin Zollanvari, Venera Nurmanova, Mehdi Bagheri

International Transactions on Electrical Energy Systems, 2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.1155/etep/7093491
Data confidence 85 high
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Transformer faults can cause power outages, equipment damage, and costly repairs, making early detection critical. Traditional diagnostic methods like dissolved gas analysis (DGA) and frequency response analysis (FRA) are useful, but they often struggle with noisy, overlapping signals, and varying operating conditions. Vibration analysis provides a noninvasive, real‐time alternative for monitoring transformer health although analyzing these signals is difficult because of external noise and complex fault patterns. The present study explores using deep learning models to predict faults from vibration time‐series data, with a focus on a hybrid architecture that combines convolutional neural networks (CNNs) and multihead self‐attention (MHSA). CNNs effectively extract local signal features, while MHSA captures long‐range dependencies and global patterns. This combination enables the model to attend to the most relevant parts of the signal and suppress noise. Interestingly, our results also reveal that when CNNs are used as feature extractors, adding the Transformer Encoder’s feedforward network (FFN) does not improve performance and can even slightly degrade it. Overall, our experiments show that the CNN–MHSA model consistently outperforms other architectures, including standalone CNNs, MHSA, Transformer Encoders, and CNN–Transformer Encoder hybrids. In terms of relative absolute error (RAE), the CNN–MHSA model achieves consistent improvements over the baseline models. In particular, the proposed model achieved an average reduction in RAE across all evaluated CNN configurations within the considered hyperparameter space, ranging from 0.5% to 1.3% for excitation voltage prediction and up to 3.7% for load current prediction—these improvements are statistically significant ( p < 0.05), as determined by a one‐sided paired t ‐test. In addition to model development, this study provides a systematic and comprehensive comparative evaluation of multiple architectures, including standalone CNNs, MHSA, Transformer Encoders, and hybrid models under consistent experimental settings. Overall, our findings demonstrate that integrating CNNs with attention mechanisms provides a powerful approach for accurate and reliable transformer fault prediction.