Adaptive Filtering for Desired Error Distribution under Minimum Information Divergence Criterion

被引:1
作者
Hu, Jinchun [1 ]
Chen, Badong [1 ]
Sun, Fuchun [1 ]
Sun, Zengqi [1 ]
机构
[1] Tsinghua Univ, State Key Lab Intelligent Technol & Syst, Dept Comp Sci & Technol, Beijing 100084, Peoples R China
来源
2008 IEEE INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1-8 | 2008年
关键词
Adaptive filtering; Information divergence; stochastic gradient algorithm; Kernel density estimation;
D O I
10.1109/IJCNN.2008.4633954
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Conventional cost functions of adaptive filtering are usually related to the error's dispersion, such as error's moments or error's entropy, but neglect the shape aspects (peaks, kurtosis, tails, etc.) of the error distribution. In this work, we propose a new notion of filtering (or estimation) in which the error's probability density function (PDF) is shaped into a desired one. As PDFs contain all the probabilistic information, the proposed method can be used to achieve the desired error variance or error entropy, and is expected to be useful in the complex signal processing and learning systems. In our approach, the information divergence between the actual errors and the desired errors is used as the cost function. By kernel density estimation, we derive the associated stochastic gradient algorithm for the finite impulse response (FIR) filter. Simulation results emphasize the effectiveness of this new algorithm in adaptive system training.
引用
收藏
页码:1215 / 1219
页数:5
相关论文
共 16 条
[1]  
[Anonymous], 1991, ELEMENT INFORM THEOR
[2]   Stochastic gradient algorithm under (h,φ)-entropy criterion [J].
Chen, B. ;
Hu, J. ;
Pu, L. ;
Sun, Z. .
CIRCUITS SYSTEMS AND SIGNAL PROCESSING, 2007, 26 (06) :941-960
[3]   Online entropy manipulation: Stochastic information gradient [J].
Erdogmus, D ;
Hild, KE ;
Principe, JC .
IEEE SIGNAL PROCESSING LETTERS, 2003, 10 (08) :242-245
[4]   Convergence properties and data efficiency of the minimum error entropy criterion in adaline training [J].
Erdogmus, D ;
Principe, JC .
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2003, 51 (07) :1966-1978
[5]   Generalized information potential criterion for adaptive system training [J].
Erdogmus, D ;
Principe, JC .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 2002, 13 (05) :1035-1044
[6]  
Haykin S., 1996, ADAPTIVE FILTERING T
[7]  
Kailath T, 2000, PR H INF SY, pXIX
[8]  
Pardo L, 2005, Statistical inference based on divergence measures
[9]   LEAST MEAN P-POWER ERROR CRITERION FOR ADAPTIVE FIR FILTER [J].
PEI, SC ;
TSENG, CC .
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 1994, 12 (09) :1540-1547
[10]  
Silverman B. W., 1986, DENSITY ESTIMATION S, DOI 10.1201/9781315140919