Information Bottleneck Approach to Predictive Inference

被引：36

作者：

Still, Susanne ^{[1
]}

机构：

[1] Univ Hawaii Manoa, Informat & Comp Sci Dept, Honolulu, HI 96822 USA

来源：

ENTROPY | 2014年 / 16卷 / 02期

关键词：

predictive inference; information bottleneck method; dynamical systems; thermodynamic efficiency; far-from-equilibrium thermodynamics; computing engines; THEORETIC APPROACH; 2ND LAW; PRINCIPLE; WORK; IRREVERSIBILITY; THERMODYNAMICS; COMPUTATION; COMPLEXITY; MECHANICS; CAPACITY;

D O I：

10.3390/e16020968

中图分类号：

O4 [物理学];

学科分类号：

0702 ;

摘要：

This paper synthesizes a recent line of work on automated predictive model making inspired by Rate-Distortion theory, in particular by the Information Bottleneck method. Predictive inference is interpreted as a strategy for efficient communication. The relationship to thermodynamic efficiency is discussed. The overall aim of this paper is to explain how this information theoretic approach provides an intuitive, overarching framework for predictive inference.

引用

页码：968 / 989

页数：22

共 73 条

[11]

Bialek W., 2001, P LES HOUCH SUMM SCH, P485

[12] COMPUTATION OF CHANNEL CAPACITY AND RATE-DISTORTION FUNCTIONS [J].

BLAHUT, RE .

IEEE TRANSACTIONS ON INFORMATION THEORY, 1972, 18 (04) :460-+

[13] Myosin V stepping mechanism [J].

Cappello, Giovanni ;

Pierobon, Paolo ;

Symonds, Clementine ;

Busoni, Lorenzo ;

Gebhardt, J. Christof M. ;

Rief, Matthias ;

Prost, Jacques .

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2007, 104 (39) :15328-15333

[14]

Chaitin GregoryJ., 2004, Algorithmic information theory, V1

[15]

Chandler D., 1987, INTRO MODERN STAT ME

[16]

Chechik G, 2005, J MACH LEARN RES, V6, P165

[17] Predictive coding and the slowness principle: An information-theoretic approach [J].

Creutzig, Felix ;

Sprekeler, Henning .

NEURAL COMPUTATION, 2008, 20 (04) :1026-1041

[18] Past-future information bottleneck in dynamical systems [J].

Creutzig, Felix ;

Globerson, Amir ;

Tishby, Naftali .

PHYSICAL REVIEW E, 2009, 79 (04)

[19]

Crooks G. E., 1999, Excursions in statistical dynamics

[20] Beyond Boltzmann-Gibbs statistics: Maximum entropy hyperensembles out of equilibrium [J].

Crooks, Gavin E. .

PHYSICAL REVIEW E, 2007, 75 (04)

← 1 2 3 4 5 6 7 8 →