Named Entities Recognition in Kazakh Text by SpaCy NER Models

2024

Closed access Metadata (CSL-JSON)
DOI 10.1109/ubmk63289.2024.10773475
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

This paper presents a study of named entity recognition (NER) in the Kazakh language using the transformer model of the spaCy library. There are a few works have been done on the NER in the Kazakh language and there is only one open data set. This indicates that the Kazakh language is a low-resource language and the relevance of the study. The article considered all scientific papers written so far on the NER in the Kazakh language and defined the main problems of subject area based on analyzing of recently published research. It provides a formal description of the transformer model from spaCy for NER. To identify the named entities from Kazakh-language texts, a data set was prepared from scientific works, and the recognition of named entities were carried out in a complete pipeline sequence. The main focus was on preparing annotated data for implementing NER from Kazakh texts and on the pipeline NER component.