Fusion-restoration model for industrial multimodal anomaly detection

被引:0
作者
Wang, Jiaxun [1 ]
Niu, Yanchang [1 ]
Huang, Biqing [1 ]
机构
[1] Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China
关键词
Anomaly detection; Multimodal fusion; Feature reconstruction; Unsupervised learning;
D O I
10.1016/j.neucom.2025.130073
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Industrial anomaly detection based on multimodal data is receiving increasing attention. The application of the feature mapping paradigm represents a prevailing trend. However, the existing feature mapping method is limited by the lack of multimodal fusion, which hinders the comprehensive interaction between RGB and point cloud features. In this paper, we introduce a novel feature reconstruction paradigm called Fusion-Restoration Model (FRM) to ameliorate this problem. A fusion encoder integrates the information of two domains into a fusion embedding. Then, a pair of decoupled decoders independently restore embeddings of the corresponding domains from the fusion embedding. FRM learns nominal feature reconstruction from anomaly-free training samples and detects and localizes anomalies based on the reconstruction residuals in the inference phase. A joint loss that constrains both direction and magnitude is used to enhance the robustness of the reconstruction. Additionally, a semi-frozen training strategy is designed to adapt the batch normalization parameters of the 3D feature extractor to the target industrial dataset. Extensive experiments show that our method achieves effective and efficient multimodal anomaly detection on the MVTec 3D-AD dataset.
引用
收藏
页数:10
相关论文
共 47 条
[1]  
Bae J, 2023, IEEE I CONF COMP VIS, P6350, DOI 10.1109/ICCV51070.2023.00586
[2]  
Bergmann P., 2021, arXiv, DOI arXiv:2112.09045
[3]   Anomaly Detection in 3D Point Clouds using Deep Geometric Descriptors [J].
Bergmann, Paul ;
Sattlegger, David .
2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2023, :2612-2622
[4]   Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings [J].
Bergmann, Paul ;
Fauser, Michael ;
Sattlegger, David ;
Steger, Carsten .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :4182-4191
[5]   MVTec AD - A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection [J].
Bergmann, Paul ;
Fauser, Michael ;
Sattlegger, David ;
Steger, Carsten .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :9584-9592
[6]   Complementary pseudo multimodal feature for point cloud anomaly detection [J].
Cao, Yunkang ;
Xu, Xiaohao ;
Shen, Weiming .
PATTERN RECOGNITION, 2024, 156
[7]   Informative knowledge distillation for image anomaly segmentation [J].
Cao, Yunkang ;
Wan, Qian ;
Shen, Weiming ;
Gao, Liang .
KNOWLEDGE-BASED SYSTEMS, 2022, 248
[8]   Emerging Properties in Self-Supervised Vision Transformers [J].
Caron, Mathilde ;
Touvron, Hugo ;
Misra, Ishan ;
Jegou, Herve ;
Mairal, Julien ;
Bojanowski, Piotr ;
Joulin, Armand .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :9630-9640
[9]  
Chen R., 2023, arXiv, DOI arXiv:2307.13925
[10]  
Chen Z., 2023, arXiv