Survival Information Potential: A New Criterion for Adaptive System Training

被引：66

作者：

Chen, Badong ^{[1
]}

Zhu, Pingping ^{[1
]}

Principe, Jose C. ^{[1
]}

机构：

[1] Univ Florida, Dept Elect & Comp Engn, Gainesville, FL 32611 USA

来源：

IEEE TRANSACTIONS ON SIGNAL PROCESSING | 2012年 / 60卷 / 03期

基金：

美国国家科学基金会; 中国国家自然科学基金;

关键词：

Minimum error entropy (MEE); supervised training; survival information potential; ENTROPY MINIMIZATION; ALGORITHM;

D O I：

10.1109/TSP.2011.2178406

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

Recently, the information potential (IP) of order, defined as the argument of the log in the alpha-order Renyi entropy, has been successfully used as an information theoretic criterion for supervised adaptive system training. In this paper, we use the survival function (or equivalently the distribution function) of an absolute value transformed random variable to define a new information potential, named the survival information potential (SIP). Compared with the IP, the SIP has some advantages, such as validity in a wide range of distributions, robustness, and the simplicity in computation. The properties of SIP and a simple formula for computing the empirical SIP are given in the paper. Finally, the SIP criterion is applied in adaptive system training, and simulation examples on FIR adaptive filtering, kernel adaptive filtering, and time delay neural networks (TDNNs) training are presented to demonstrate the performance.

引用

页码：1184 / 1194

页数：11

共 22 条

[1] Stochastic gradient algorithm under (h,φ)-entropy criterion [J].

Chen, B. ;

Hu, J. ;

Pu, L. ;

Sun, Z. .

CIRCUITS SYSTEMS AND SIGNAL PROCESSING, 2007, 26 (06) :941-960

[2] Δ-Entropy: Definition, properties and applications in system identification with quantized data [J].

Chen, Badong ;

Zhu, Yu ;

Hu, Jinchun ;

Principe, Jose C. .

INFORMATION SCIENCES, 2011, 181 (07) :1384-1402

[3] BLOCK IMPLEMENTATION OF ADAPTIVE DIGITAL-FILTERS [J].

CLARK, GA ;

MITRA, SK ;

PARKER, SR .

IEEE TRANSACTIONS ON ACOUSTICS SPEECH AND SIGNAL PROCESSING, 1981, 29 (03) :744-752

[4]

Cover T.M., 1991, ELEMENT INFORM THEOR

[5] Generalized information potential criterion for adaptive system training [J].

Erdogmus, D ;

Principe, JC .

IEEE TRANSACTIONS ON NEURAL NETWORKS, 2002, 13 (05) :1035-1044

[6] An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems [J].

Erdogmus, D ;

Principe, JC .

IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2002, 50 (07) :1780-1786

[7] From linear adaptive filtering to nonlinear information processming [J].

Erdogmus, Deniz ;

Principe, Jose C. .

IEEE SIGNAL PROCESSING MAGAZINE, 2006, 23 (06) :14-33

[8]

Kaplan D., 1995, Understanding Nonlinear Dynamics

[9]

Kuo J. M., 1993, THESIS U FLORIDA GAI

[10] The kernel least-mean-square algorithm [J].

Liu, Weifeng ;

Pokharel, Puskal P. ;

Principe, Jose C. .

IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2008, 56 (02) :543-554

← 1 2 3 →