Multilevel Annotation Model for Detecting Fake News in Kazakh-Language Media

2025

Closed access Metadata (CSL-JSON)
DOI 10.1109/ubmk67458.2025.11206945
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

With the exponential growth of disinformation in digital media, especially in multilingual societies, there is an increasing need to develop formalized annotation models that can accurately reflect both the content and the pragmatic structure of fake news. The present study presents a multi-level annotation scheme focused on Kazakh language media content and aimed at improving the interpretability and automatibility of fake news detection processes. Unlike the dominant binary approaches (REAL/FAKE), the developed model includes nine key categories (including CLAIM, EVIDENCE, SOURCE, AUTHOR_INTENT, TARGET_AUDIENCE, etc.) and twenty-three subtypes reflecting manipulative techniques, author's motivations, and the typology of target audiences. The manually annotated corpus of 5,000 texts covers news reports in Kazakh and demonstrates high values of inter - editorial agreement (k = 0.72-0.89). The annotation methodology is implemented in Label Studio with a further formalized description of the semantic relationships between entities. The results of the study emphasize both the scientific novelty of the proposed scheme and its applied significance in the tasks of constructing explicable models in the field of media literacy, information security and linguistically adapted NLP.