Abstract
This paper reports a systematic study of low-rank adaptation (LoRA)-based fine-tuning applied to Qwen3 language models (4B, 8B, and 14B parameters) for the task of legal question answering within the jurisdiction of the Republic of Kazakhstan. The bilingual dataset comprises 63,114 question–answer pairs (76.2% Russian, 23.8% Kazakh) covering 11 legal domains. Models are evaluated through both automated metrics (BERTScore, citation accuracy, and hallucination rate) and blind expert assessment by a panel of two practising legal experts. Key findings: (1) all fine-tuned models reach BERTScore F1 close to 90% (89.6–90.2%) versus 82.2–83.1% for untuned base models; (2) fine-tuned models outperform GPT-4o (87.2%) and GPT-4o-mini (86.7%) on semantic similarity while exhibiting far lower hallucination rates (27–29% vs. 83–90%); (3) blind expert assessment confirms the advantage of fine-tuned models, with panel mean completeness scores of 4.28/5 versus 1.95/5 for base models (quadratically weighted Cohen’s κ = 0.80–0.95 across rating dimensions, indicating substantial to almost perfect inter-rater agreement); and (4) we identify a practical scaling plateau: paired Wilcoxon tests (n = 500) detect statistically significant but practically small differences across the 4B, 8B, and 14B fine-tuned variants (largest mean gap 0.67 pp on BERTScore F1; Cohen’s |d_z| ≤ 0.34), gains too small to justify the 3.5× parameter increase. These findings show that parameter-efficient adaptation of compact open-source models can match or exceed commercial LLMs for specialised legal QA in a low-resource bilingual context. We note one scope restriction: three domains (administrative, criminal, and housing law) are represented only in Russian, so the model is not validated for Kazakh language queries in these areas.