Empirical Estimation of Information Measures: A Literature Guide

被引：35
作者：
Verdu, Sergio
机构：
[1] Princeton, 08540, NJ
来源：
ENTROPY | 2019年 / 21卷 / 08期
基金：
美国国家科学基金会;
关键词：
information measures; empirical estimators; entropy; relative entropy; mutual information; universal estimation; DIVERGENCE ESTIMATION; ENTROPY ESTIMATION; NONPARAMETRIC-ESTIMATION; CONVERGENCE PROPERTIES; MUTUAL INFORMATION; RELATIVE ENTROPY; FUNCTIONALS; DISTRIBUTIONS; CLASSIFICATION; DENSITIES;
D O I：
10.3390/e21080720
中图分类号：
O4 [物理学];
学科分类号：
0702 ;
摘要：
We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in data science, machine learning, biology, neuroscience, economics, language, and other experimental sciences.
引用
收藏
页数：16
相关论文
共 131 条
[1]  
Acharya J, 2017, PR MACH LEARN RES, V70
[2]   Information theory in molecular biology [J].
Adami, Christoph .
PHYSICS OF LIFE REVIEWS, 2004, 1 (01) :3-22
[3]  
Adams TM, 1998, ANN PROBAB, V26, P794
[4]   NONPARAMETRIC ESTIMATION OF ENTROPY FOR ABSOLUTELY CONTINUOUS DISTRIBUTIONS [J].
AHMAD, IA ;
LIN, PE .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1976, 22 (03) :372-375
[5]  
[Anonymous], 2016, ARXIV160600304
[6]  
[Anonymous], 2016, ARXIV160509124
[7]  
[Anonymous], 2018, ARXIV180208405
[8]  
[Anonymous], ARXIV180501355
[9]  
[Anonymous], 1987, PROBLEMY PEREDACHI I
[10]  
[Anonymous], 1968, INFORM THEORY STAT
12345678910