Extracting Explanations, Justification, and Uncertainty from Black-Box Deep Neural Networks

被引:0
作者
Ardis, Paul [1 ]
Flenner, Arjuna [2 ]
机构
[1] GE Aerosp Res, 1 Res Circle, Niskayuna, NY 12309 USA
[2] GE Aerosp, 3290 Patterson Ave SE, Grand Rapids, MI 49512 USA
来源
ASSURANCE AND SECURITY FOR AI-ENABLED SYSTEMS | 2024年 / 13054卷
关键词
D O I
10.1117/12.3012765
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Deep Neural Networks (DNNs) do not inherently compute or exhibit empirically-justified task confidence. In mission critical applications, it is important to both understand associated DNN reasoning and its supporting evidence. In this paper, we propose a novel Bayesian approach to extract explanations, justifications, and uncertainty estimates from DNNs. Our approach is efficient both in terms of memory and computation, and can be applied to any black box DNN without any retraining, including applications to anomaly detection and out-of-distribution detection tasks. We validate our approach on the CIFAR-10 dataset, and show that it can significantly improve the interpretability and reliability of DNNs.
引用
收藏
页数:8
相关论文
共 50 条
[41]   Revisiting Black-box Ownership Verification for Graph Neural Networks [J].
Zhou, Ruikai ;
Yang, Kang ;
Wang, Xiuling ;
Wang, Wendy Hui ;
Xu, Jun .
45TH IEEE SYMPOSIUM ON SECURITY AND PRIVACY, SP 2024, 2024, :2478-2496
[42]   Feature Importance Explanations for Temporal Black-Box Models [J].
Sood, Akshay ;
Craven, Mark .
THIRTY-SIXTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FOURTH CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE / TWELVETH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2022, :8351-8360
[43]   Uncertainty propagation method for high-dimensional black-box problems via Bayesian deep neural network [J].
Jing Fei Liu ;
Chao Jiang ;
Jing Zheng .
Structural and Multidisciplinary Optimization, 2022, 65
[44]   Comparing Explanations from Glass-Box and Black-Box Machine-Learning Models [J].
Kuk, Michal ;
Bobek, Szymon ;
Nalepa, Grzegorz J. .
COMPUTATIONAL SCIENCE - ICCS 2022, PT III, 2022, 13352 :668-675
[45]   Uncertainty propagation method for high-dimensional black-box problems via Bayesian deep neural network [J].
Liu, Jing Fei ;
Jiang, Chao ;
Zheng, Jing .
STRUCTURAL AND MULTIDISCIPLINARY OPTIMIZATION, 2022, 65 (03)
[46]   Local Explanations and Self-Explanations for Assessing Faithfulness in black-box LLMs [J].
Fragkathoulas, Christos ;
Chlapanis, Odysseas S. .
PROCEEDINGS OF THE 13TH HELLENIC CONFERENCE ON ARTIFICIAL INTELLIGENCE, SETN 2024, 2024,
[47]   Compressing Deep Neural Network: A Black-Box System Identification Approach [J].
Sahu, Ishan ;
Pal, Arpan ;
Ukil, Arijit ;
Majumdar, Angshul .
2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
[48]   AdverseGen: A Practical Tool for Generating Adversarial Examples to Deep Neural Networks Using Black-Box Approaches [J].
Zhang, Keyuan ;
Wu, Kaiyue ;
Chen, Siyu ;
Zhao, Yunce ;
Yao, Xin .
ARTIFICIAL INTELLIGENCE XXXVIII, 2021, 13101 :313-326
[49]   A Black-Box Attack on Neural Networks Based on Swarm Evolutionary Algorithm [J].
Liu, Xiaolei ;
Hu, Teng ;
Ding, Kangyi ;
Bai, Yang ;
Niu, Weina ;
Lu, Jiazhong .
INFORMATION SECURITY AND PRIVACY, ACISP 2020, 2020, 12248 :268-284
[50]   SOTER: Guarding Black-box Inference for General Neural Networks at the Edge [J].
Shen, Tianxiang ;
Qi, Ji ;
Jiang, Jianyu ;
Wang, Xian ;
Wen, Siyuan ;
Chen, Xusheng ;
Zhao, Shixiong ;
Wang, Sen ;
Chen, Li ;
Luo, Xiapu ;
Zhang, Fengwei ;
Cui, Heming .
PROCEEDINGS OF THE 2022 USENIX ANNUAL TECHNICAL CONFERENCE, 2022, :723-737