Аңдатпа
The article presents the results of the development and pilot testing of an adaptive learning system designed to develop and assess students’ critical thinking using elements of artificial intelligence. The scientific novelty of the study, in comparison with existing research on automated essay scoring and LLM-based assessment, lies in the multi-component assessment of critical thinking, the cascade mechanism of AI-based assessment, and the appeal mechanism. These features make it possible to more accurately identify students’ weak points in the development of critical thinking, ensure transparent assessment of students’ critical thinking, increase the system’s resilience to failures, and provide students with the opportunity to challenge the assessment in controversial situations. The system evaluates essays and problem-solving tasks according to six components of critical thinking: analysis, interpretation, evaluation, argumentation, logical coherence, and reflection. For each component, calculation formulas were proposed; these are computed using language models, semantic embeddings based on sentence-transformers, and a backup TF-IDF mechanism. The pilot testing was conducted on a sample of 20 students from Satbayev University. Three instructors, represented by the authors, participated in the expert assessment. The experimental data included 40 student works: 20 essays and 20 problem-solving tasks. Each work was independently assessed by three instructors as well as by the learning system. The reference score was determined as the average score of the three instructors. The correlation between the automated assessment and the averaged expert assessment was r = 0.84, while the mean deviation was 1.0 point on a ten-point scale. The results obtained indicate that the system can be used as a tool to support instructors; however, further research on a larger sample is required. Gamification elements, progress visualization, and the appeal mechanism increase students’ engagement and ensure a balance between automated and expert assessment.