Evaluating Multimodal LLMs for Context-Aware Forensic Image Interpretation

Taras Fedynyshyn, Olha Mykhaylova, Ivan Opirskyy, Olzhas Konakbayev

2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.1007/s44227-026-00102-6
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Abstract Digital forensics is vital for analyzing extensive image data from mobile devices to identify individuals and activities in investigations. Traditional methods struggle with complex real-world images, particularly distinguishing military personnel from military-themed mannequins. This study assesses multimodal Large Language Models (LLMs) - Google’s Gemini 1.5 Pro, open-source LLAVA, and GPT-4o - for detecting military personnel in 434 mobile device images, including military personnel, mannequins, and civilians. The models achieved strong recall (0.99 for Gemini, 0.98 for LLAVA, and 0.91 for GPT-4o) but only moderate precision (0.69, 0.69, and 0.67 respectively), reflecting a notable rate of mannequin-induced false positives.] Accuracy varied from 0.793 for Gemini and LLAVA to 0.770 for GPT-4o, aligning with observed differences in contextual understanding. Contextual classification also posed challenges: Gemini achieved 0.787 accuracy for country identification, followed by GPT-4o (0.385) and LLAVA (0.121). Unit name recognition remained weak across models. Misclassification of mannequins was the primary source of error, confirming that current multimodal models overemphasize uniform and equipment cues without verifying human authenticity. To enhance interpretability and reduce false positives, we integrated an agentic orchestration layer using CrewAI and LangGraph, which structured multimodal reasoning through dedicated sub-agents for provenance validation, perception, mannequin discrimination, and evidence-grounded attribution. These agentic frameworks substantially improved forensic reliability: CrewAI achieved 0.88 precision with a mannequin false-positive rate of 0.12, while LangGraph reached 0.90 precision and reduced false positives to 0.08. Country attribution accuracy rose to 0.58 and 0.62 respectively. Although recall decreased slightly due to conservative abstention logic (CrewAI 0.74, LangGraph 0.73), this trade-off yielded higher forensic confidence and reproducible, audit-ready decision traces. The results demonstrate that integrating agentic architectures transforms multimodal LLMs from opaque classifiers into transparent, evidence-driven forensic tools - enhancing both analytic precision and the evidentiary defensibility of AI-assisted investigations.