An end-to-end approach to detect railway track defects based on supervised and self-supervised learning

•Development of an end-to-end deep learning framework for detecting defects on railway tracks using YOLOv8 and U-Net models.•The proposed YOLOv8-Segment model outperforms other segmentation models in terms of precision, recall, and mean average precision (mAP) for defects related to railway track components.•Implementation of U-Net based self-supervised model to overcome the challenge of scarce labeled data for rail surface defects.•Comparative analysis of the proposed approach with state-of-the-art methods, demonstrating superior performance on both custom and benchmark datasets. Railway infrastructure is critical for the safe and efficient transportation of goods and people. However, defects in rail tracks can lead to severe accidents, resulting in significant human and financial losses. This study presents the development of an end-to-end railway track defect detection system utilizing advanced deep learning techniques. The proposed system is designed to detect various defects, including rail surface anomalies and component defects such as missing fasteners, bolts, and fishplates. The approach combines a supervised YOLOv8x Segment Model and a self-supervised U-Net model. The YOLOv8x-segment model, trained on a curated dataset comprising 3,500 images annotated with five classes - rail, fastener, defective fastener, fish-plate and defective fishplate. The model achieved a mean Average Precision (mAP) of 95 % and demonstrated its capability for real-time detection of rails, fasteners and fishplates with a high inference speed of 40 FPS. To overcome the issue of scarcity of labeled rail surface defect dataset, the U-Net model was pre-trained on 400 normal rail images and was fine-tuned using a subset of defect dataset. This approach resulted in significant improvements in defect segmentation accuracy, effectively detecting rail surface anomalies without relying on extensive labeled datasets. The system demonstrated strong performance across various metrics and datasets, providing a reliable tool for enhancing railway safety through automated defect detection. Future work includes expanding detection capabilities to include sleepers and ballast quantity, further enhancing the system's applicability in comprehensive railway infrastructure monitoring.