Rate-Distortion Theory by and for Energy-Based Models

被引：0

作者：

Li, Qing ^{[1
]}

Guyot, Cyril ^{[1
,2
]}

机构：

[1] Western Digital Res, Milpitas, CA 95035 USA

[2] Alibaba Cloud, Sunnyvale, CA 94085 USA

来源：

IEEE TRANSACTIONS ON COMMUNICATIONS | 2024年 / 72卷 / 07期

关键词：

Rate-distortion; Noise reduction; Distortion; Neural networks; Training; Electronics packaging; Source coding; Rate-distortion theory; energy-based models; batch denoising; Blahut-Arimoto algorithm;

D O I：

10.1109/TCOMM.2024.3361531

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

In this work, we examine the relationship between rate-distortion theory and energy-based models (EBMs). We demonstrate that EBMs can be used to approximate the rate-distortion approaching posterior, as in the Blahut-Arimoto (BA) algorithm, and to solve batch denoising problems using the posterior distribution learned by EBMs. Our results highlight the potential of EBMs to enhance the efficiency of rate-distortion theory analysis and vice versa.

引用

页码：4072 / 4083

页数：12

共 47 条

[1]

[Anonymous], 2006, Predicting Structured Data

[2] ALGORITHM FOR COMPUTING CAPACITY OF ARBITRARY DISCRETE MEMORYLESS CHANNELS [J].

ARIMOTO, S .

IEEE TRANSACTIONS ON INFORMATION THEORY, 1972, 18 (01) :14-+

[3]

Ba J, 2014, ACS SYM SER

[4]

Bengio Y, 2001, ADV NEUR IN, V13, P932

[5] COMPUTATION OF CHANNEL CAPACITY AND RATE-DISTORTION FUNCTIONS [J].

BLAHUT, RE .

IEEE TRANSACTIONS ON INFORMATION THEORY, 1972, 18 (04) :460-+

[6]

Bora A, 2017, PR MACH LEARN RES, V70

[7] StarGAN v2: Diverse Image Synthesis for Multiple Domains [J].

Choi, Yunjey ;

Uh, Youngjung ;

Yoo, Jaejun ;

Ha, Jung-Woo .

2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, :8185-8194

[8]

Cover TM., 2006, ELEMENTS INFORM THEO, DOI [DOI 10.1002/047174882X, DOI 10.1002/0471200611.CH2]

[9] On the Tchebychef inequality of Bernstein [J].

Craig, CC .

ANNALS OF MATHEMATICAL STATISTICS, 1933, 4 :94-102

[10]

Csiszar Imre, 1982, Information theory: coding theorems for discrete memoryless systems

← 1 2 3 4 5 →