A Multilevel Visual and Textual Framework for Near-Duplicate Diagram Detection in Electronic Documents

Svitlana Biloshchytska, Oleksandr Kuchanskyi, Yurii Andrashko, Zhan Amangeldiyev, Dina Kantayeva, Myroslava Tovt-Kuchanska

Information, 2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.3390/info17090897
Data confidence 85 high
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Near-duplicate diagram detection in electronic documents is challenging because diagram identity depends on graphical structure, spatial composition, and textual labels, while reused images may undergo compression, cropping, rotation, photometric changes, or perspective distortion. This study proposes a cascaded multimodal framework combining perceptual hashing, Siamese Residual Network with 18 layers (Siamese ResNet18), Distillation with No Labels, ver. 2 (DINOv2) visual representations, and a text-similarity classifier. A controlled benchmark was constructed from Artificial Intelligence 2D Diagram Dataset (AI2D) using Light, Medium, and Hard transformations, with source-grouped splitting by base_id to prevent leakage across training, validation, and test sets. Perceptual hashing achieved Area Under the Receiver Operating Characteristic Curve (ROC-AUC) = 0.7483, while Siamese ResNet18 increased ROC-AUC to 0.8599. DINOv2 provided the strongest visual performance, achieving Accuracy = 0.9933, F1-score = 0.9933, ROC-AUC = 0.9992, and Average Precision = 0.9994; F1-score remained 0.9901 for Hard transformations. Visual fusion increased ROC-AUC to 0.9995, and full multimodal fusion reached ROC-AUC = 0.9998. At an early-exit threshold of 0.95, 27.6% of pairs were resolved at the hashing level. These results support the coarse-to-fine design on the constructed AI2D-derived benchmark. A targeted hard-negative stress test revealed substantially higher false-positive rates under deliberately matched spatial layouts, with an overall False Positive Rate (FPR) of 0.48 for DINOv2 and 0.16 for full multimodal fusion. Generalization to naturally reused or redrawn diagrams, larger and more diverse hard-negative collections, and Optical Character Recognition (OCR)-derived text remains to be evaluated.