Abstract
Named entity transcription remains a challenge for low-resource languages such as Kazakh, often causing errors in proper nouns, locations, and other named entities. This paper presents a fine-tuning approach using OpenAI’s Whisper model, KazakhTTS2 and KazEmoTTS text-to-speech models for synthetic speech generation, KazNERD, a named entity recognition dataset for Kazakh, and the Kazakh Speech Corpus as a source of real transcribed speech. Whisper was fine-tuned on a dataset with real speech from the Kazakh Speech Corpus and synthetic speech from KazNERD, generated using KazakhTTS2 and KazEmoTTS text-to-speech models. Additional synthetic data were created by modifying named entities in KazNERD sentences using ChatGPT before synthesis. Evaluation using word error rate, character error rate, and named entity word error rate shows improved named entity transcription while maintaining overall automatic speech recognition performance. This approach enhances Kazakh automatic speech recognition and provides a scalable method for named entity recognition in low-resource languages.