The ''independent components'' of natural scenes are edge filters

被引:1484
作者
Bell, AJ
Sejnowski, TJ
机构
[1] Howard Hughes Medical Institute, Compl. Neurobiology Laboratory, Salk Institute, San Diego, CA 92037
关键词
information theory; independent components; neural network learning;
D O I
10.1016/S0042-6989(97)00121-1
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
It has previously been suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representation of natural scenes, and it has been reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features, We show here that a new unsupervised learning algorithm based on information maximization, a nonlinear ''infomax'' network, when applied to an ensemble of natural scenes produces sets of visual filters that are localized and oriented, Some of these filters are Gabor-like and resemble those produced by the sparseness-maximization network. In addition, the outputs of these filters are as independent as possible, since this infomax network performs Independent Components Analysis or ICA, for sparse (super-gaussian) component distributions, We compare the resulting ICA filters and their associated basis functions, with other decorrelating filters produced by Principal Components Analysis (PCA) and zero-phase whitening filters (ZCA), The ICA filters have more sparsely distributed (kurtotic) outputs on natural scenes, They also resemble the receptive fields of simple cells in visual cortex, which suggests that these neurons form a natural, information-theoretic coordinate system for natural images. (C) 1997 Elsevier Science Ltd.
引用
收藏
页码:3327 / 3338
页数:12
相关论文
共 54 条
[1]  
AMARI S, 1996, ADV NEURAL INFORMATI, V8
[2]  
[Anonymous], P EUR SIGN PROC C BR
[3]   Could information theory provide an ecological theory of sensory processing? [J].
Aticky, Joseph J. .
NETWORK-COMPUTATION IN NEURAL SYSTEMS, 2011, 22 (1-4) :4-44
[4]   CONVERGENT ALGORITHM FOR SENSORY RECEPTIVE-FIELD DEVELOPMENT [J].
ATICK, JJ ;
REDLICH, AN .
NEURAL COMPUTATION, 1993, 5 (01) :45-60
[5]   Towards a Theory of Early Visual Processing [J].
Atick, Joseph J. ;
Redlich, A. Norman .
NEURAL COMPUTATION, 1990, 2 (03) :308-320
[6]  
BADDELEY R, 1996, IN PRESS NETWORK
[7]   Unsupervised Learning [J].
Barlow, H. B. .
NEURAL COMPUTATION, 1989, 1 (03) :295-311
[8]  
BARLOW HB, 1992, OPTICAL SOC AM TECHN, V23, P172
[9]  
BARLOW HB, 1994, LARGE SCALE NEURONAL
[10]   AN INFORMATION MAXIMIZATION APPROACH TO BLIND SEPARATION AND BLIND DECONVOLUTION [J].
BELL, AJ ;
SEJNOWSKI, TJ .
NEURAL COMPUTATION, 1995, 7 (06) :1129-1159