Context-aware KazRoBERTa with transformer-based candidate re-ranking for Kazakh spelling correction

Zhanar Azhibekova, Daniyar Sultan, Laura M. Baitenova

Frontiers in Artificial Intelligence, 2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.3389/frai.2026.1925050
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Automatic spelling correction for Kazakh is difficult because productive morphology, vowel harmony, and Cyrillic keyboard substitutions generate many orthographically plausible alternatives. This study proposes Context-Aware KazRoBERTa, a three-stage framework that combines token-level error detection, linguistically constrained candidate generation, and masked-language-model candidate re-ranking with a shared Transformer encoder. Candidate generation integrates inverse ka2ru mapping, vowel-harmony restoration, Optimal String Alignment distance, and lexicon filtering; contextual probability, orthographic similarity, and lexical frequency are then combined for selection. Evaluation was performed on a synthetic corpus of 2,615,771 parallel sentence pairs derived from 922,798 clean sentences. On the held-out synthetic test set, the framework achieved 97.7% detection F1-score and 98.8% detection accuracy, while sentence correction accuracy reached 85.8%, word correction accuracy reached 95.6%, and character error rate decreased to 1.86%. The 95% Wilson interval for sentence correction accuracy was 85.36–86.23%. Relative to the reported two-stage baseline, the 5.6-point sentence-accuracy margin implies a conservative continuity-corrected McNemar lower bound of χ 2 > = 230.3 ( p < 0.001) across all paired contingency tables compatible with the reported rounded margins. Ablation results attribute the largest gain to contextual re-ranking, particularly for deletion and substitution errors. The evidence nevertheless remains limited to one fixed-seed synthetic evaluation: no natural-error test corpus, direct ByT5/mT5/NLLB or sequence-tagging baseline, paired prediction archive, repeated-run variance, or measured latency benchmark was supplied. The web interface therefore demonstrates system integration rather than external accuracy. The scientific contribution is the Kazakh-specific, parameter-sharing hybrid design; claims of real-world generalization or state-of-the-art superiority require external benchmarking and fully archived configurations.