Abstract
Since its launch in November 2022, chatbot ChatGPT has gained significant popularity worldwide. It performs the task of a search engine, analyzes the information, and generates the required output. ChatGPT is already recognized as a useful tool for educational purposes, but it also comes with some limitations and potential risks. In this case study, we tested and analyzed ChatGPT’s ability to solve typical physics tests and problems in four languages: Kazakh, Latvian, Russian, and English. We collected over 400 questions, including problem-solving tasks, to evaluate ChatGPT’s performance. Compared to students' results, ChatGPT performed slightly better on multiple-choice theory questions but struggled with practical problem-solving, achieving only 17% correct answers. The highest accuracy was observed in English, with an 83% success rate, while Kazakh had the lowest at just 33%, showing the problems of using ChatGPT in non-English languages. The potential for unfair use of ChatGPT and measures to address this issue are also briefly discussed.