Abstract
In this paper, we present a review of the latest deep learning methods used in inspecting railway infrastructure with vision systems. Traditional methods for inspecting railways are not always efficient, scalable or accurate, so using computer vision and artificial intelligence is now needed to maintain rail safety. The research divides more than 200 papers into important themes, covering rail surface fault detection, issues with fasteners and sleepers, adding synthetic data, using different types of railroad sites and multi-technology integration. For every domain, we cover current progressive models, for example, CNNs, YOLOv5, U-Net, Faster R-CNN and Vision Transformers, discussing their uses, pros and cons. The problem of using the model in practice is analyzed by studying generalization, instant results, understanding how it learns and the hardware limitations. We also find areas where progress is needed and suggest future solutions to reduce the difference between what academic discovery and what industries can use.