Information Bottleneck Approach to Predictive Inference

被引：36
作者：
Still, Susanne [1 ]
机构：
[1] Univ Hawaii Manoa, Informat & Comp Sci Dept, Honolulu, HI 96822 USA
来源：
ENTROPY | 2014年 / 16卷 / 02期
关键词：
predictive inference; information bottleneck method; dynamical systems; thermodynamic efficiency; far-from-equilibrium thermodynamics; computing engines; THEORETIC APPROACH; 2ND LAW; PRINCIPLE; WORK; IRREVERSIBILITY; THERMODYNAMICS; COMPUTATION; COMPLEXITY; MECHANICS; CAPACITY;
D O I：
10.3390/e16020968
中图分类号：
O4 [物理学];
学科分类号：
0702 ;
摘要：
This paper synthesizes a recent line of work on automated predictive model making inspired by Rate-Distortion theory, in particular by the Information Bottleneck method. Predictive inference is interpreted as a strategy for efficient communication. The relationship to thermodynamic efficiency is discussed. The overall aim of this paper is to explain how this information theoretic approach provides an intuitive, overarching framework for predictive inference.
引用
收藏
页码：968 / 989
页数：22
相关论文
共 73 条
[1]  
[Anonymous], 2006, Elements of Information Theory
[2]  
[Anonymous], 1993, 31 ANN M ASS COMPUTA, DOI [10.3115/981574.981598, DOI 10.3115/981574.981598]
[3]  
[Anonymous], 1998, THEORY PROBABILITY
[4]  
[Anonymous], 2012, BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHING BIOPHYSICS SEARCHI
[5]  
[Anonymous], 1993, Predictive inference
[6]  
[Anonymous], 1984, The Dripping Faucet as a Model Chaotic System
[7]  
[Anonymous], ADV COMPLEX SYSTEMS
[8]  
[Anonymous], 1973, Gravitation
[9]   ALGORITHM FOR COMPUTING CAPACITY OF ARBITRARY DISCRETE MEMORYLESS CHANNELS [J].
ARIMOTO, S .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1972, 18 (01) :14-+
[10]   Predictability, complexity, and learning [J].
Bialek, W ;
Nemenman, I ;
Tishby, N .
NEURAL COMPUTATION, 2001, 13 (11) :2409-2463
12345678