Abstract
The increasing adoption of Internet of Things (IoT) devices necessitates efficient synchronization methods to ensure reliable communication and coordination. This paper introduces an autonomous approach to one-to-many beacon synchronization using Reinforcement Learning (RL) to adjust clock-sleep cycles for very imprecise-clock devices operating in energy-constrained IoT environments. The proposed method tackles key challenges such as clock drift and acute time fluctuations, which complicate synchronization and lead to unexpected desynchronizations. The novelty of this approach lies not in the method used to adjust sleep cycles but in its autonomous, device-specific adaptation, eliminating the need for manual calibration or model training. Through extensive experimentation with popular ESP32 devices, we evaluate the performance of the proposed RL-based solution against traditional beacon synchronization and supervised learning techniques, demonstrating improvements in synchronization adaptiveness and energy efficiency, achieving energy savings of up to 4.5 mJ per round. Furthermore, we introduce a knowledge transfer mechanism that enables newly joined devices to rapidly adapt their synchronization parameters based on the experiences of previously trained devices. To encourage reproducibility, we provide open-source implementations of the proposed framework.