A CLASS OF RENYI INFORMATION ESTIMATORS FOR MULTIDIMENSIONAL DENSITIES

被引：147

作者：

Leonenko, Nikolai ^{[1
]}

Pronzat, Luc ^{[2
]}

Savani, Vippal ^{[1
]}

机构：

[1] Cardiff Univ, Cardiff Sch Math, Cardiff CF24 4AG, S Glam, Wales

[2] Univ Nice, CNRS, Lab 13S, F-06903 Sophia Antipolis, France

来源：

ANNALS OF STATISTICS | 2008年 / 36卷 / 05期

基金：

英国工程与自然科学研究理事会;

关键词：

Entropy estimation; estimation of statistical distance; estimation of divergence; nearest-neighbor distances; Renyi entropy; Havrda-Charvat entropy; Tsallis entropy;

D O I：

10.1214/07-AOS539

中图分类号：

O21 [概率论与数理统计]; C8 [统计学];

学科分类号：

020208 ; 070103 ; 0714 ;

摘要：

A class of estimators of the Renyi and Tsallis entropies of an unknown distribution f in R-m is presented. These estimators are based on the kth nearest-neighbor distances computed from a sample of N i.i.d. vectors with distribution f. We show that entropies of any order q, including Shannon's entropy, can be estimated consistently with minimal assumptions on f. Moreover, we show that it is straightforward to extend the nearest-neighbor method to estimate the statistical distance between two distributions using one i.i.d. sample from each.

引用

页码：2153 / 2182

页数：30

共 46 条

[1]

ALEMANY P, 1992, PHYS REV E, V49, P956

[2]

[Anonymous], 1992, MULTIVARIATE DENSITY

[3]

[Anonymous], 1020 IRISA

[4]

[Anonymous], 2003, Journal of Machine Learning Research

[5]

[Anonymous], MATH METHODS STAT

[6]

[Anonymous], 1989, Maximum-entropy models in science and engineering

[7]

Beirlant J, 1997, INT J MATH STAT SCI, V6, P17

[8] SUMS OF FUNCTIONS OF NEAREST NEIGHBOR DISTANCES, MOMENT BOUNDS, LIMIT-THEOREMS AND A GOODNESS OF FIT TEST [J].

BICKEL, PJ ;

BREIMAN, L .

ANNALS OF PROBABILITY, 1983, 11 (01) :185-214

[9]

BIERENS H, 1994, TOPICS ADV ECONOMETR, DOI DOI 10.1017/CBO9780511599279

[10] STRONG UNIFORM CONSISTENCY OF NEAREST NEIGHBOR DENSITY ESTIMATES [J].

DEVROYE, LP ;

WAGNER, TJ .

ANNALS OF STATISTICS, 1977, 5 (03) :536-540

← 1 2 3 4 5 →