DIVERGENCE MEASURES BASED ON THE SHANNON ENTROPY

被引：3013
作者：
LIN, JH
机构：
[1] Department of Computer Science, Brandeis University, Waltham
来源：
IEEE TRANSACTIONS ON INFORMATION THEORY | 1991年 / 37卷 / 01期
关键词：
D O I：
10.1109/18.61115
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
A new class of information-theoretic divergence measures based on the Shannon entropy is introduced. Unlike the well-known Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds. These bounds are crucial in many applications of divergence measures. The new measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness. © 1991 IEEE
引用
收藏
页码：145 / 151
页数：7
相关论文
共 32 条
[1]  
ACZEL J, 1975, MEASURES INFORMATION
[2]  
ALI SM, 1966, J ROY STAT SOC B, V28, P131
[3]  
BASSAT MB, 1978, INFORM CONTR, V39, P227
[4]  
Chen C. H., 1973, STAT PATTERN RECOGNI
[5]   INFORMATION AND DISTANCE MEASURES, ERROR BOUNDS, AND FEATURE SELECTION [J].
CHEN, CH .
INFORMATION SCIENCES, 1976, 10 (02) :159-173
[6]   APPROXIMATING DISCRETE PROBABILITY DISTRIBUTIONS WITH DEPENDENCE TREES [J].
CHOW, CK ;
LIU, CN .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1968, 14 (03) :462-+
[7]  
Csiszar I., 1967, STUDIA SCI MATH HUNG, V2, P299
[8]  
GALLAGER RG, 1968, INFORMATION THEORY R
[9]  
GOKHALE DV, 1978, INFORMATION CONTINGE
[10]  
GUIASU S, 1977, INFORMATION THEORY A
1234