Estimating entropy on m bins given fewer than m samples

被引：79

作者：

Paninski, L ^{[1
]}

机构：

[1] UCL, Gatsby Computat Neurosci Unit, London WC1N 3AR, England

来源：

IEEE TRANSACTIONS ON INFORMATION THEORY | 2004年 / 50卷 / 09期

关键词：

approximation theory; bias; consistency; distribution-free bounds; entropy; estimation;

D O I：

10.1109/TIT.2004.833360

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Consider a sequence p(N) of discrete probability measures, supported on m(N) points, and assume that we observe N independent and identically distributed (i.i.d.) samples from each p(N). We demonstrate the existence of an estimator of the entropy, H (p(N)),which is consistent even if the ratio N/m(N) is bounded (and, as a corollary, even if this ratio tends to zero, albeit at a sufficiently slow rate).

引用

页码：2200 / 2203

页数：4

共 10 条

[1] [Anonymous], 1961, CONTRIBUTIONS THEORY
[2] [Anonymous], 1955, INFORM THEORY PSYCHO
[3] Convergence properties of functional estimates for discrete distributions
Antos, A
Kontoyiannis, I
[J]. RANDOM STRUCTURES & ALGORITHMS, 2001, 19 (3-4) : 163 - 193
[4] A MEASURE OF ASYMPTOTIC EFFICIENCY FOR TESTS OF A HYPOTHESIS BASED ON THE SUM OF OBSERVATIONS
CHERNOFF, H
[J]. ANNALS OF MATHEMATICAL STATISTICS, 1952, 23 (04): : 493 - 507
[5] Devroye L., 1996, A probabilistic theory of pattern recognition
[6] DONOHO DL, 1991, ANN STAT, V19, P633, DOI 10.1214/aos/1176348114
[7] Koosis P., 1988, LOGARITHMIC INTEGRAL
[8] LeCam L. M., 1986, ASYMPTOTIC METHODS S
[9] Estimation of entropy and mutual information
Paninski, L
[J]. NEURAL COMPUTATION, 2003, 15 (06) : 1191 - 1253
[10] Rudin W., 1973, FUNCTIONAL ANAL

← 1 →