Normal-Knowledge-Based Pavement Defect Segmentation Using Relevance-Aware and Cross-Reasoning Mechanisms

Automatic pavement defect segmentation is a big challenge because of the class diversity and extremely random distribution of defects. Most existing approaches focus on supervised strategies to achieve decent performance. Due to the difficulty of getting massive densely annotated samples and the limited prior knowledge of potential defects, these methods have significant bottlenecks in the actual pavement settings. This paper proposes a relevance-aware and cross-reasoning network (RCN) for anomaly segmentation of pavement defects, which can segment defects using merely non-defective images for training. A relevance-aware transformer-based encoder is first devised to model intrinsic interdependencies across local features, thus improving representations of complex non-defective images. Next, a dual decoder strategy is proposed to remap the encoder-generated latent dependencies at the local semantic and global detailed levels, respectively. Specifically, a cross-reasoning refinement module is built in the local decoder to reason the cross-relationship between spatial and channel dimensions. Finally, a context-aware abnormal distillation measurement is developed to evaluate the semantic reconstruction deviations during the inference. Under the guidance of semantic affinity, this measurement allows our model to highlight defective areas adaptively. Extensive experimental results on four datasets indicate that RCN outperforms other leading anomaly segmentation methods.