Scalable pipelines for instant object detection: deploying YOLO models with Apache Kafka for high-throughput inference

2025

Full text Open access: Hybrid Metadata (CSL-JSON)
DOI 10.54596/2958-0048-2025-2-194-206
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Instant object detection is a critical capability in modern applications where timely decision-making is essential, such as in emergency medicine, autonomous systems, and intelligent surveillance. Efficiently handling high-throughput image streams with minimal delay presents significant challenges, particularly under demanding conditions. This study presents a scalable object detection pipeline that integrates YOLO models with Apache Kafka, a distributed streaming platform, to support just-in-time inference. The proposed architecture leverages Kafka’s partitioning and consumer group mechanisms to enable parallel processing, ensuring high throughput without requiring complex load-balancing logic. The system is deployed on a Virtual Private Server to demonstrate practical implementation. Two configurations are presented to illustrate Kafka’s native scalability: one with a single partition and a single consumer, and another with five partitions and five consumers. These setups visually demonstrate how Kafka efficiently distributes workloads across multiple consumers. Although specific latency or throughput metrics are not reported, the architecture effectively showcases how Kafka’s design enables prompt responses to high-volume input. This pipeline is well-suited for time-sensitive object detection tasks and can be extended to a wide range of instant analytics applications where rapid feedback is critical.