Entropy Bounds for Discrete Random Variables via Maximal Coupling

被引:30
作者
Sason, Igal [1 ]
机构
[1] Technion Israel Inst Technol, Dept Elect Engn, IL-32000 Haifa, Israel
基金
以色列科学基金会;
关键词
Coupling; entropy; local distance; Stein's method; total variation distance; GEOMETRIC APPROXIMATION; MUTUAL INFORMATION; POISSON APPROXIMATION; STEINS METHOD; INEQUALITIES; DISTANCE; DISTRIBUTIONS; PROBABILITY; INTERPLAY; LAW;
D O I
10.1109/TIT.2013.2274515
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal coupling, and they apply to discrete random variables which are defined over finite or countably infinite alphabets. Loosened versions of these bounds are demonstrated to reproduce some previously reported results. The use of the new bounds is exemplified for the Poisson approximation, where bounds on the local and total variation distances follow from Stein's method.
引用
收藏
页码:7118 / 7131
页数:14
相关论文
共 38 条
  • [1] Abramowitz M., 1964, HDB MATH FUNCTIONS, V55
  • [2] Sharp Bounds on the Entropy of the Poisson Law and Related Quantities
    Adell, Jose A.
    Lekuona, Alberto
    Yu, Yaming
    [J]. IEEE TRANSACTIONS ON INFORMATION THEORY, 2010, 56 (05) : 2299 - 2306
  • [3] [Anonymous], 2006, Elements of Information Theory
  • [4] [Anonymous], 2007, 2 COURSE PROBABILITY
  • [5] [Anonymous], INT MATH FORUM
  • [6] [Anonymous], 1981, Information Theory: Coding Theorems for Discrete Memoryless Systems
  • [7] [Anonymous], ENTROPY SEARCH COMPL
  • [8] 2 MOMENTS SUFFICE FOR POISSON APPROXIMATIONS - THE CHEN-STEIN METHOD
    ARRATIA, R
    GOLDSTEIN, L
    GORDON, L
    [J]. ANNALS OF PROBABILITY, 1989, 17 (01) : 9 - 25
  • [9] Compound Poisson Approximation via Information Functionals
    Barbour, A. D.
    Johnson, O.
    Kontoyiannis, I.
    Madiman, M.
    [J]. ELECTRONIC JOURNAL OF PROBABILITY, 2010, 15 : 1344 - 1368
  • [10] Barbour A.D., 1992, Poisson approximation