Advancing Machine-generated Text Detection: A Comprehensive Evaluation of Transformer-based Models

Journal of Advances in Information Technology, 2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.12720/jait.17.7.1321-1330
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Improved fluency in large language models has intensified the need for accurate detection of machine-generated text.This study evaluates transformer-based models using an improved version of the Conference on Computational Linguistics 2025 (COLING 2025), Generative Artificial Intelligence (GenAI) Content Detection Task 1 dataset, which was carefully preprocessed to enhance label quality and balance.All models were trained under a unified protocol to ensure fair comparison and robust evaluation.Test set results show that Decoding-Enhanced Bert with Disentangled Attention (DeBERTa) achieves the highest macro F1-Score of 85.48%, surpassing the previously top-ranked Multi-Task Learning (MTL) system, which attains a macro F1 of 83.07%.These results highlight the effectiveness of advanced transformer architectures for distinguishing human-written and machine-generated text.Despite these gains, performance degradation under domain shift and highly paraphrased inputs remains a challenge.