Fisher information and stochastic complexity

被引:528
作者
Rissanen, JJ
机构
[1] IBM ARC, Almadcn Research Center, San Jose
关键词
universal coding; universal modeling; MDL principle;
D O I
10.1109/18.481776
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
By taking into account the Fisher information and removing an inherent redundancy in earlier two-part codes, a sharper code length as the stochastic complexity and the associated universal process are derived for a class of parametric processes. The main condition required is that the maximum-likelihood estimates satisfy the Central Limit Theorem, The same code length is also obtained from the so-called maximum-likelihood code.
引用
收藏
页码:40 / 47
页数:8
相关论文
共 22 条
[1]  
Barron A. R., 1985, THESIS STANFORD U ST
[2]  
BERNARDO JM, 1979, J R STAT SOC B, V41, P113
[3]   INFORMATION-THEORETIC ASYMPTOTICS OF BAYES METHODS [J].
CLARKE, BS ;
BARRON, AR .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1990, 36 (03) :453-471
[4]   JEFFREYS PRIOR IS ASYMPTOTICALLY LEAST FAVORABLE UNDER ENTROPY RISK [J].
CLARKE, BS ;
BARRON, AR .
JOURNAL OF STATISTICAL PLANNING AND INFERENCE, 1994, 41 (01) :37-60
[5]   UNIVERSAL NOISELESS CODING [J].
DAVISSON, LD .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1973, 19 (06) :783-795
[6]   MINIMAX NOISELESS UNIVERSAL CODING FOR MARKOV SOURCES [J].
DAVISSON, LD .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1983, 29 (02) :211-215
[7]   A SOURCE MATCHING APPROACH TO FINDING MINIMAX CODES [J].
DAVISSON, LD ;
LEONGARCIA, A .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1980, 26 (02) :166-174
[8]  
ELIAS P, 1975, IEEE T INFORM THEORY, V21, P194, DOI 10.1109/TIT.1975.1055349
[9]   THE PERFORMANCE OF UNIVERSAL ENCODING [J].
KRICHEVSKY, RE ;
TROFIMOV, VK .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1981, 27 (02) :199-207
[10]  
LJUNG L, 1979, STOCHASTICS, V17, P29