Automatic Railroad Track Components Inspection Using Hybrid Deep Learning Framework

Regular inspections on track components, such as the clip, spike, and rail, are essential to maintain track quality and ensure railroad operating safety. Unfortunately, traditional image processing (IP)-based systems have limited accuracy. Existing convolutional neural network (CNN)-based approaches are designed for either detection or saliency segmentation of a specific track component (e.g., fastener or rail only). The overall track condition could not be evaluated because not all the track components are inspected simultaneously. This article presents an all-in-one YOLO (AOYOLO) framework for multitask track component inspection. First, a newly developed ConvNeXt-based backbone is constructed in AOYOLO to produce suitable hyperfeatures for both detection and segmentation tasks. Second, a novel U-shaped salient object segmentation branch is incorporated into AOYOLO to supplement the object detection branch, improving both the rail surface defect (RSD) segmentation and the detection of other components. Advanced data augmentations are integrated to further enhance the accuracy and scalability of the network. Extensive experiments conducted on a track dataset established with images taken by drone indicate that the proposed system is able to: 1) achieve 95.6% mean average precision (mAP) for track components inspection at a real-time speed of 147 frames/s and 2) reach 93.6% accuracy on RSDs detection, which surpass the current state-of-the-art (SOTA) models. The exceptional inference speed and superior detection accuracy have great potential for field applications.