Self-supervised anomaly detection and localization for X-ray cargo images: Generalization to novel anomalies

Robust detection of illicit items using X-ray inspection methods has gained increasing importance in recent years due to the large volume of cargo crossing international borders. In addition to detecting the presence of such items, determining their location, size, and shape is challenging due to the unpredictable nature of anomalies, but essential for expediting security inspections. Viewing the illicit items as anomalies relative to expected cargo, we propose a self-supervised learning framework consisting of an encoder-decoder-classifier- segmenter model, a multi-component loss function, coupled with a training strategy to extract discriminative features tailored for detection of the presence of anomalies, as well as localization of such items in X-ray cargo images. Our framework addresses the challenges posed by limited labeled data and offers a model capable of both detecting and localizing anomalies effectively. Moreover, we present a diverse dataset encompassing various cargo scenarios with and without anomalies, providing a robust evaluation environment for this class of problems. Unlike existing approaches, which are trained to detect specific types of objects with a fixed set of illicit items, our framework is adaptable to real-world scenarios where a wide range of illicit items may be present in the cargo. This versatility enhances the practical applicability of our model. We evaluate the performance of our framework on our dataset as well as two other publicly available datasets, demonstrating our method's strong detection and localization performance even when faced with complex novel anomalies significantly different from those encountered during training.