Deploying Multilingual ASR in Digital Twin Systems: A Performance and Efficiency Analysis of Whisper and SeamlessM4T

2025

Closed access Metadata (CSL-JSON)
DOI 10.1109/acdsa65407.2025.11166030
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

This study presents a systematic comparison of two leading automatic speech recognition (ASR) model families—OpenAI's Whisper and Meta's SeamlessM4T—across three typologically diverse languages: English (Germanic), Russian (Slavic), and Kazakh (Turkic). Motivated by the need to identify optimal ASR solutions for multilingual collaborative environments in digital twin development, we evaluate these models using standardized benchmarks including LibriSpeech and Common Voice datasets. Our analysis employs Word Error Rate (WER), Character Error Rate (CER), and GPU memory utilization as primary metrics. Results demonstrate that Whisper large-v3 achieves the best performance for English and Russian with WERs of 3.29% and 9.83% respectively, while SeamlessM4T-v2-large offers superior performance for Kazakh with a WER of 13.7%. GPU memory requirements range from 144.4MB to 6025.7MB, scaling with model size. These findings provide valuable guidance for selecting appropriate ASR models for multilingual applications in collaborative environments where accurate voice transcription is essential for human-machine interfaces in digital twin systems.