Abstract
This work investigates the use of ResNet-based AEs to reduce the impact of laser phase noise within coherent optical communication transmission. Traditional AE designs usually rely on dense layers, which are simple to implement but struggle with the complex structure of fiber-optic signals. To address this limitation, this work introduces a ResNet-based AE architecture for both the transmitter and receiver. The model is trained end-to-end to learn constellations and decoding strategies that are more robust to phase noise. Simulation results using 16-QAM show that the proposed ResNet-based design consistently outperforms dense-layer AEs in terms of AIR and BER, especially at larger laser linewidths. The results demonstrate that the residual structure enables deeper and more expressive feature learning, providing stronger resilience to laser phase noise compared to conventional dense architectures.