Abstract
Streaming systems often struggle with memory overload when large amounts of data are processed in real time. This paper presents a lightweight Information Lifecycle Management (ILM) strategy designed to control hot-tier memory usage in Redis by dynamically archiving stale records to cold storage. The proposed method combines time-based expiration with an asynchronous sweeper thread that migrates inactive keys after a configurable threshold. The approach was implemented and tested within a Kafka-Spark-Redis pipeline over$\mathbf{3 0}$-minute workloads. The results show that Redis memory usage was reduced by 83 %, falling from 8.1 GB to just 1.4 GB, with only a 1 % decrease in Kafka throughput and a$\mathbf{1 \%}$increase in Spark latency. These findings demonstrate that effective memory control can be achieved without compromising real-time performance. The architecture remains fully containerized and uses only native TTL and file operations, making it easily deployable in resourceconstrained environments.