On the Opacity of Deep Neural Networks

被引:2
作者
Sogaard, Anders [1 ]
机构
[1] Univ Copenhagen, Ctr Philosophy AI, Copenhagen, Denmark
关键词
deep neural networks; opacity; explainability; model size; mitigation;
D O I
10.1017/can.2024.1
中图分类号
B [哲学、宗教];
学科分类号
01 ; 0101 ;
摘要
Deep neural networks are said to be opaque, impeding the development of safe and trustworthy artificial intelligence, but where this opacity stems from is less clear. What are the sufficient properties for neural network opacity? Here, I discuss five common properties of deep neural networks and two different kinds of opacity. Which of these properties are sufficient for what type of opacity? I show how each kind of opacity stems from only one of these five properties, and then discuss to what extent the two kinds of opacity can be mitigated by explainability methods.
引用
收藏
页码:224 / 239
页数:16
相关论文
共 53 条
[1]   Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging [J].
Arun, Nishanth ;
Gaw, Nathan ;
Singh, Praveer ;
Chang, Ken ;
Aggarwal, Mehak ;
Chen, Bryan ;
Hoebel, Katharina ;
Gupta, Sharut ;
Patel, Jay ;
Gidwani, Mishka ;
Adebayo, Julius ;
Li, Matthew D. ;
Kalpathy-Cramer, Jayashree .
RADIOLOGY-ARTIFICIAL INTELLIGENCE, 2021, 3 (06)
[2]  
Askari A., 2020, 23 INT C ARTIFICIAL
[3]  
Aytekin C, 2022, Arxiv, DOI arXiv:2210.05189
[4]  
Ba LJ, 2014, ADV NEUR IN, V27
[5]   Beware explanations from AI in health care [J].
Babic, Boris ;
Gerke, Sara ;
Evgeniou, Theodoros ;
Cohen, I. Glenn .
SCIENCE, 2021, 373 (6552) :284-286
[6]   Opacity thought through: on the intransparency of computer simulations [J].
Beisbart, Claus .
SYNTHESE, 2021, 199 (3-4) :11643-11666
[7]   Two Dimensions of Opacity and the Deep Learning Predicament [J].
Boge, Florian J. .
MINDS AND MACHINES, 2022, 32 (01) :43-75
[8]  
Bucilu C., 2006, P ACM INT C KNOWL DI, P535, DOI DOI 10.1145/1150402.1150464
[9]   How the machine 'thinks': Understanding opacity in machine learning algorithms [J].
Burrell, Jenna .
BIG DATA & SOCIETY, 2016, 3 (01) :1-12
[10]   Discrete and continuous representations and processing in deep learning: Looking forward [J].
Cartuyvels, Ruben ;
Spinks, Graham ;
Moens, Marie-Francine .
AI OPEN, 2021, 2 :143-159