Limitations of Deep Learning for Inverse Problems on Digital Hardware

被引：9

作者：

Boche, Holger ^{[1
,2
,3
,4
]}

Fono, Adalbert ^{[5
]}

Kutyniok, Gitta ^{[6
,7
]}

机构：

[1] Tech Univ Munich, Inst Theoret Informat Technol, D-80333 Munich, Germany

[2] Ruhr Univ Bochum, BMBF Res Hub 6G Life Excellence Cluster Cyber Secu, D-44801 Bochum, Germany

[3] Munich Ctr Quantum Sci & Technol MCQST, D-80799 Munich, Germany

[4] Munich Quantum Valley MQV, D-80807 Munich, Germany

[5] Ludwig Maximilians Univ Munchen, Dept Math, D-80539 Munich, Germany

[6] Ludwig Maximilians Univ Munchen, Dept Math, D-80539 Munich, Germany

[7] Munich Ctr Machine Learning MCML, D-80538 Munich, Germany

来源：

IEEE TRANSACTIONS ON INFORMATION THEORY | 2023年 / 69卷 / 12期

关键词：

Neural networks; Computational modeling; Hardware; Inverse problems; Deep learning; Approximation algorithms; Task analysis; Computing theory; deep learning; signal processing; turing machine; THRESHOLDING ALGORITHM; ADVERSARIAL ATTACKS; RECURSIVE FUNCTIONS; NON-COMPUTABILITY; SIGNAL RECOVERY; NEURAL-NETWORKS; SPARSE SIGNALS; RECONSTRUCTION; CIRCUIT;

D O I：

10.1109/TIT.2023.3326879

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Deep neural networks have seen tremendous success over the last years. Since the training is performed on digital hardware, in this paper, we analyze what actually can be computed on current hardware platforms modeled as Turing machines, which would lead to inherent restrictions of deep learning. For this, we focus on the class of inverse problems, which, in particular, encompasses any task to reconstruct data from measurements. We prove that finite-dimensional inverse problems are not Banach-Mazur computable for small relaxation parameters. Even more, our results introduce a lower bound on the accuracy that can be obtained algorithmically.

引用

页码：7887 / 7908

页数：22

共 89 条

[1] The Gap between Theory and Practice in Function Approximation with Deep Neural Networks [J].

Adcock, Ben ;

Dexter, Nick .

SIAM JOURNAL ON MATHEMATICS OF DATA SCIENCE, 2021, 3 (02) :624-655

[2] Solving ill-posed inverse problems using iterative deep neural networks [J].

Adler, Jonas ;

Oktem, Ozan .

INVERSE PROBLEMS, 2017, 33 (12)

[3] Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey [J].

Akhtar, Naveed ;

Mian, Ajmal .

IEEE ACCESS, 2018, 6 :14410-14430

[4]

[Anonymous], 1902, Bulletin of the American Mathematical Society, DOI DOI 10.1090/S0002-9904-1902-00923-3

[5] On instabilities of deep learning in image reconstruction and the potential costs of AI [J].

Antun, Vegard ;

Renna, Francesco ;

Poon, Clarice ;

Adcock, Ben ;

Hansen, Anders C. .

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2020, 117 (48) :30088-30095

[6] Deep-learning tomography [J].

Araya-Polo, Mauricio ;

Jennings, Joseph ;

Adler, Amir ;

Dahlke, Taylor .

Leading Edge, 2018, 37 (01) :58-66

[7] Solving inverse problems using data-driven models [J].

Arridge, Simon ;

Maass, Peter ;

Oktem, Ozan ;

Schonlieb, Carola-Bibiane .

ACTA NUMERICA, 2019, 28 :1-174

[8]

Avigad J, 2014, LECT NOTES LOGIC, V42, P1

[9]

Bastounis A, 2022, Arxiv, DOI arXiv:2110.15734

[10]

Bastounis A, 2021, Arxiv, DOI arXiv:2109.06098

← 1 2 3 4 5 6 7 8 9 →