Abstract
The Kazakh language, with its agglutinative morphology and scarce annotated data, presents challenges for accurate Question Answering (QA). This study proposes a three-stage framework to improve multilingual QA performance in low-resource settings through model adaptation, semantic representation evaluation, and retrieval optimization. First, multilingual QA models (mT5, XLM-R, mDeBERTa-v3, AYA, Kaz-RoBERTa, and KazakhBERTmulti) were fine-tuned using SentencePiece tokenization and adapter-based training. Second, five embedding models were benchmarked via Domain and QuestionType classification, with Snowflake-Arctic achieving the highest accuracy (0.87) and BGE-M3 demonstrating strong robustness. Finally, a memory-aware retrieval mechanism was implemented by integrating Safe Memory clustering with FAISS indexing, improving contextual recall (R@10 = 0.984) without loss of precision. Results show that adapted multilingual transformers consistently outperform native Kazakh models, while enhanced retrieval strengthens grounding and cross-domain robustness. The framework provides a reproducible pathway toward scalable and linguistically inclusive QA systems for low-resource languages.