An analysis of entropy estimators for blind source separation

被引:34
作者
Hild, KE
Erdogmus, D
Principe, JC [1 ]
机构
[1] Univ Florida, Dept Elect & Comp Engn, Gainesville, FL 32611 USA
[2] Univ Calif San Francisco, Dept Radiol, San Francisco, CA 94122 USA
[3] Oregon Hlth Sci Univ, Dept Comp Sci & Engn, Beaverton, OR 97006 USA
[4] Oregon Hlth Sci Univ, Dept Biomed Engn, Beaverton, OR 97006 USA
基金
美国国家科学基金会;
关键词
blind source separation; information theoretic learning; Renyi's quadratic entropy; kurtosis; independent component analysis;
D O I
10.1016/j.sigpro.2005.04.015
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
An extensive analysis of a non-parametric, information-theoretic method for instantaneous blind source separation (BSS) is presented. As a result a modified stochastic information gradient estimator is proposed to reduce the computational complexity and to allow the separation of sub-Gaussian sources. Interestingly, the modification enables the method to simultaneously exploit spatial and spectral diversity of the sources. Consequently, the new algorithm is able to separate i.i.d. sources, which requires higher-order spatial statistics, and it is also able to separate temporally correlated Gaussian sources, which requires temporal statistics. Three reasons are given why Renyi's entropy estimators for Information-Theoretic Learning (ITL), on which the proposed method is based, is to be preferred over Shannon's entropy estimators for ITL. Also contained herein is an extensive comparison of the proposed method with JADE, Infomax, Comon's MI, FastICA, and a non-parametric, information-theoretic method that is based on Shannon's entropy. Performance comparisons are shown as a function of the data length, source kurtosis, number of sources, and stationarity/correlation of the sources. (c) 2005 Published by Elsevier B.V.
引用
收藏
页码:182 / 194
页数:13
相关论文
共 27 条
[1]  
AMARI S, 1996, ADV NEURAL INFORMATI, P127
[2]   AN INFORMATION MAXIMIZATION APPROACH TO BLIND SEPARATION AND BLIND DECONVOLUTION [J].
BELL, AJ ;
SEJNOWSKI, TJ .
NEURAL COMPUTATION, 1995, 7 (06) :1129-1159
[3]   Efficiency of high-order moment estimates [J].
Bourin, C ;
Bondon, P .
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 1998, 46 (01) :255-258
[4]   Infomax and maximum likelihood for blind source separation [J].
Cardoso, JF .
IEEE SIGNAL PROCESSING LETTERS, 1997, 4 (04) :112-114
[5]   BLIND BEAMFORMING FOR NON-GAUSSIAN SIGNALS [J].
CARDOSO, JF ;
SOULOUMIAC, A .
IEE PROCEEDINGS-F RADAR AND SIGNAL PROCESSING, 1993, 140 (06) :362-370
[6]   Blind signal separation: Statistical principles [J].
Cardoso, JF .
PROCEEDINGS OF THE IEEE, 1998, 86 (10) :2009-2025
[7]   Equivariant adaptive source separation [J].
Cardoso, JF ;
Laheld, BH .
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 1996, 44 (12) :3017-3030
[8]   INDEPENDENT COMPONENT ANALYSIS, A NEW CONCEPT [J].
COMON, P .
SIGNAL PROCESSING, 1994, 36 (03) :287-314
[9]  
Cover T., 1991, ELEMENTS INFORM THEO, P20
[10]   Online entropy manipulation: Stochastic information gradient [J].
Erdogmus, D ;
Hild, KE ;
Principe, JC .
IEEE SIGNAL PROCESSING LETTERS, 2003, 10 (08) :242-245