Development of a Framework for Analyzing Kazakh Language Errors in Semi-Structured Data

2025

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.1016/j.procs.2025.08.260
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

This paper presents a novel hybrid framework for systematic error analysis in semi-structured Kazakh texts, addressing the dual challenges of rich agglutinative morphology and low-resource data environments. The proposed pipeline integrates deterministic linguistic rules derived from the Apertium Turkic lexicon with probabilistic classifiers trained on an iteratively expanded corpus of 120 198 sentences. Core components include text normalization, tokenization, lexicon lookup, morphological parsing, and error classification, each formalized through mathematical functions to ensure reproducibility. Empirical evaluation against baseline systems demonstrated an F₁-score improvement of 12 percentage points, with preprocessing accuracy rising from 89 % to 96 % following successive lexicon augmentations. Cross-platform analysis of 3 000 records revealed that Russian-spelling interference accounts for nearly half of all detected errors, whereas typographical and pure spelling mistakes vary substantially across user-generated and editorial sources. Comprehensive linguistic resources—comprising over 70 000 root forms, 5 136 affix entries, and a validated correct-word dictionary—were compiled to underpin robust error detection. While manual lexicon enrichment proved effective, its labor intensity suggests the need for semi-supervised morphological induction in future work. Overall, this study establishes a scalable, adaptable approach for error analysis in agglutinative languages and lays the groundwork for extending these methods to other Turkic language contexts.