A Multi-criteria-Based Expert Modeling for Assessing the Textbook Quality

2023

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.18178/ijiet.2023.13.9.1944
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

This research considers the application of a multi-criteria approach to assess the quality of textbook content. The issues of determining evaluation criteria and organizing the work of experts are brought to the fore, as these factors directly affect the objectivity and efficiency of the evaluation process. The definition of criteria consisted of reviewing common mistakes and comments on the quality of textbooks, forming a preliminary and final list of criteria. The organization of the work of the experts consisted in the development of a platform with the introduction of a multi-criteria approach. As a result, 39 criteria were formulated and proposed, grouped into 10 main sections such as “Structure and organization”, “Content”, “Didactic aspects”, “Methodological aspects”, “Language level”, “Psychological and psycholinguistic aspects”, “Cultural and Value Aspects”, “ICT Integration”, “Differentiation Aspects”, and “Layout and Design”. The work of experts in the developed expert system is described, which helps in making a decision. The implementation of the multi-criteria approach in this expert system is the main contribution of the research work, which helps to improve the process of evaluating textbooks. The paper presents the results of the survey among 44 experts that helped to identify the most significant criteria which affect the evaluation a textbook. The paper presents the results of the survey among 44 experts that helped to identify the most significant criteria which affect the evaluation a textbook. Moreover, the base of allowed errors was formulated for the following types: for illustrations, QR codes, for texts, for the language of presentation, technical errors, content errors, errors for multimedia objects.