Developing Students’ HOTS Digital Literacy Profile by Using IRT

Begimbetova Guldana Atymtaevna, Heri Retnawati, Gulzhaina Kuralbayevna Kassymova, Mochamad Bruri Triyono, Mohamed Nor Azhari Azman, Reisa Aulia Sodikin

2025

Data confidence 85 high

How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

This study aimed to develop and validate a digital literacy higher-order thinking skills (HOTS) cognitive test for engineering students at Universitas Negeri Yogyakarta in Indonesia. Utilising a quantitative research design, the study involved 95 participants and employed expert judgement to assess content validity through the Aiken V Index, which yielded a high average validity score of 1.783. Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) confirmed a single factor structure with strong fit indices (CFI = 0.98, TLI = 0.97, RMSEA = 0.08), indicating robust construct validity. Reliability analysis revealed a high internal consistency (α = 0.92). Item Response Theory (IRT) calibration demonstrated satisfactory model fit, with moderate discrimination and average difficulty indices for the test items. The findings affirm that the developed test is both reliable and valid, effectively capturing students' digital literacy HOTS profiles, thereby providing valuable insights for educational interventions aimed at enhancing digital literacy skills in engineering education.