Statistical mechanics of lossy data compression using a nonmonotonic perceptron

被引:80
作者
Hosaka, T [1 ]
Kabashima, Y
Nishimori, H
机构
[1] Tokyo Inst Technol, Dept Computat Intelligence & Syst Sci, Yokohama, Kanagawa 2268502, Japan
[2] Tokyo Inst Technol, Dept Phys, Tokyo 1528551, Japan
来源
PHYSICAL REVIEW E | 2002年 / 66卷 / 06期
关键词
D O I
10.1103/PhysRevE.66.066126
中图分类号
O35 [流体力学]; O53 [等离子体物理学];
学科分类号
070204 ; 080103 ; 080704 ;
摘要
The performance of a lossy data compression scheme for uniformly biased Boolean messages is investigated via methods of statistical mechanics. Inspired by a formal similarity to the storage capacity problem in neural network research, we utilize a perceptron of which the transfer function is appropriately designed in order to compress and decode the messages. Employing the replica method, we analytically show that our scheme can achieve the optimal performance known in the framework of lossy compression in most cases when the code length becomes infinite. The validity of the obtained results is numerically confirmed.
引用
收藏
页码:8 / 066126
页数:8
相关论文
共 31 条
[1]  
Aji S, 2001, IMA V MATH, V123, P195
[2]   STORAGE CAPACITY AND GENERALIZATION ERROR FOR THE REVERSED-WEDGE ISING PERCEPTRON [J].
BEX, GJ ;
SERNEELS, R ;
VANDENBROECK, C .
PHYSICAL REVIEW E, 1995, 51 (06) :6309-6312
[3]  
Cover T. M., 2005, ELEM INF THEORY, DOI 10.1002/047174882X
[4]   STABILITY OF SHERRINGTON-KIRKPATRICK SOLUTION OF A SPIN GLASS MODEL [J].
DEALMEIDA, JRL ;
THOULESS, DJ .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1978, 11 (05) :983-990
[5]   OPTIMAL STORAGE PROPERTIES OF NEURAL NETWORK MODELS [J].
GARDNER, E ;
DERRIDA, B .
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1988, 21 (01) :271-284
[6]  
HOSAKA T, 2002, THESIS TOKYO I TECHN
[7]  
INOUE J, CONDMAT9806037
[8]  
Jelinek F., 1968, PROBABILISTIC INFORM
[9]   Statistical mechanics of error-correcting codes [J].
Kabashima, Y ;
Saad, D .
EUROPHYSICS LETTERS, 1999, 45 (01) :97-103
[10]   Typical performance of Gallager-type error-correcting codes [J].
Kabashima, Y ;
Murayama, T ;
Saad, D .
PHYSICAL REVIEW LETTERS, 2000, 84 (06) :1355-1358