Convolutional Neural Networks Analyzed via Convolutional Sparse Coding

被引:0
作者
Papyan, Vardan [1 ]
Romano, Yaniv [2 ]
Elad, Michael [1 ]
机构
[1] Technion Israel Inst Technol, Dept Comp Sci, IL-32000 Haifa, Israel
[2] Technion Israel Inst Technol, Dept Elect Engn, IL-32000 Haifa, Israel
基金
欧洲研究理事会; 以色列科学基金会;
关键词
Deep Learning; Convolutional Neural Networks; Forward Pass; Sparse Representation; Convolutional Sparse Coding; Thresholding Algorithm; Basis Pursuit; SIGNAL RECOVERY; DICTIONARY; REPRESENTATIONS;
D O I
暂无
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Convolutional neural networks (CNN) have led to many state-of-the-art results spanning through various fields. However, a clear and profound theoretical understanding of the forward pass, the core algorithm of CNN, is still lacking. In parallel, within the wide field of sparse approximation, Convolutional Sparse Coding (CSC) has gained increasing attention in recent years. A theoretical study of this model was recently conducted, establishing it as a reliable and stable alternative to the commonly practiced patch-based processing. Herein, we propose a novel multi-layer model, ML-CSC, in which signals are assumed to emerge from a cascade of CSC layers. This is shown to be tightly connected to CNN, so much so that the forward pass of the CNN is in fact the thresholding pursuit serving the ML-CSC model. This connection brings a fresh view to CNN, as we are able to attribute to this architecture theoretical claims such as uniqueness of the representations throughout the network, and their stable estimation, all guaranteed under simple local sparsity conditions. Lastly, identifying the weaknesses in the above pursuit scheme, we propose an alternative to the forward pass, which is connected to deconvolutional and recurrent networks, and also has better theoretical guarantees.
引用
收藏
页码:1 / 52
页数:52
相关论文
共 77 条
[1]  
[Anonymous], 2017, ARXIV170108349
[2]  
[Anonymous], 2010, P ADV NEUR INF PROC
[3]  
[Anonymous], 2016, ARXIV160303417
[4]  
[Anonymous], WAVELET TOUR SIGNAL
[5]  
[Anonymous], 2015, Very Deep Convolu- tional Networks for Large-Scale Image Recognition
[6]  
[Anonymous], 2015, arXiv
[7]  
[Anonymous], 2016, ARXIV160308155
[8]  
[Anonymous], 2013, ARXIV13126120
[9]  
[Anonymous], 2015, AISTATS
[10]  
[Anonymous], 2015, PROC CVPR IEEE