Benchmarking LLM Backends for Generative SSH Honeypots: Security, Fidelity, Hallucination, Latency, and Stability

AI, 2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.3390/ai7080287
Data confidence 85 high
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Large language model (LLM) backends increasingly generate SSH-honeypot output, yet the literature fixes one backend per system and judges realism by human evaluators, never asking which LLM is fit to play the shell or measuring the catastrophic failure of a backend leaking its own instructions. We fix one hardened unprivileged user scaffold and prompt, vary only the backend across eleven LLMs, and replace the human judge with an objective, prompt-anchored leakage metric. Across a controlled 42-command battery (20 trials each; 8736 responses) and a live adaptive corpus (8626 responses), we score six dimensions: instruction leakage, fidelity, hallucination, latency, verbosity, and stability. Exactly one backend (gemma-4-31b-it) reproduces verbatim leakage in both datasets and is disqualified; the other ten never leak. A deterministic handler layer serves 33 of 42 commands identically across backends, confining model risk to nine generative commands, where flag fabrication ranges 0–95%, latency spans an order of magnitude, and a held-out classifier identifies the backend from one response at 71% versus 9% chance, a fingerprinting risk. We report a per-dimension scorecard rather than a weight-sensitive ranking. Honeypot safety on the privilege boundary is a property of the architecture; realism, speed, cost, and one catastrophic leak are properties of the model.