Integration AI Techniques in Low-Resource Language: The Case of Kazakh Language

Kamshat Asmaganbetova, Ulanbek Auyeskhan, Nurzhanova Zarina, Serikbayev Alen, Malika Yeskazina

2024

Closed access Metadata (CSL-JSON)
DOI 10.1109/ieeeconf61558.2024.10585350
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Sentiment analysis is an extensively explored domain within natural language processing (NLP); nevertheless, a significant emphasis has been placed on languages possessing ample resources, such as English. This paper delves into the transformative capabilities of Optical Character Recognition (OCR), Natural Language Processing (NLP), and Speech Recognition, emphasizing their potential impact on low-resource languages, particularly focusing on Kazakh. OCR, a pivotal AI technique, extracts characters from images, revolutionizing the digitization of business records and streamlining mundane tasks. NLP, another powerful AI tool, analyzes and generates human-like text, while Speech Recognition converts spoken words into text. According to findings the OCR model demonstrated effectiveness, achieving an accuracy rate of 85% with losses ranging from 1.5 to 2, while TTS (text-to-speech) method mean opinion score (MOS) results with 95% confidence intervals. In addition, AI Meta research center assessed over 40,000 diverse translations, including Kazakh language directions using the human-translated benchmark, they integrated human evaluation with a unique toxicity benchmark across all 200 languages to ensure translation safety. Achieving a substantial 44% improvement over the prior state-of-the-art, the model by Meta AI establishes a crucial foundation for realizing a universal translation system. However, there remains ample room for exploration and refinement in the integration of these AI technologies, particularly in expanding the scope of evaluation metrics and addressing potential challenges in low-resource language settings.