UCMR-Net: Text-Anchored Residual Fusion with Adaptive Residual Weighting for Multimodal Sentiment Intensity Prediction

Daoyun Tang, Gulshat A. Amirkhanova, Yanwei Fu

Applied Sciences, 2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.3390/app16147142
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Robust multimodal sentiment analysis requires models that can integrate linguistic, acoustic, and visual cues while avoiding over-reliance on noisy nonverbal signals. This study proposes UCMR-Net, a text-anchored residual fusion framework for continuous multimodal sentiment intensity prediction. The model uses contextual textual representations as the primary semantic backbone and introduces acoustic and visual representations as adaptive residual correction signals. Instead of treating the learned positive residual coefficient as a direct estimate of aleatoric or epistemic uncertainty, the proposed framework interprets it as a residual reliability score for regulating nonverbal contribution. Under a unified five-seed evaluation protocol on CMU-MOSI, UCMR-Net achieves MAE = 0.699 ± 0.009, RMSE = 0.997 ± 0.011, Pearson correlation = 0.802 ± 0.006, Acc-2 = 85.82 ± 0.62%, and F1 = 85.60 ± 0.62% (mean ± SD). Controlled ablation results show that text anchoring is the dominant contributor to regression improvement, while residual fusion, adaptive residual weighting, counterfactual distillation, and multi-task supervision provide secondary stabilization effects. Unified missing-modality evaluation further indicates that performance remains relatively stable when audio or visual streams are removed, but degrades substantially when text is unavailable, confirming that the model is text-anchored rather than modality-symmetric. Calibration analysis shows that raw UCMR-Net only modestly improves calibration-related metrics, whereas post-hoc temperature scaling reduces ECE from 0.121 to 0.064 and NLL from 2.337 to 2.286. Additional CMU-MOSEI validation suggests that the proposed fusion strategy generalizes beyond CMU-MOSI, although cross-dataset transfer remains more challenging. Overall, UCMR-Net provides an effective and empirically validated framework for complete-modality sentiment intensity prediction, with moderate robustness under nonverbal missing or corrupted conditions.