Rate-Distortion Theory by and for Energy-Based Models

被引：0

作者：

Li, Qing ^{[1
]}

Guyot, Cyril ^{[1
,2
]}

机构：

[1] Western Digital Res, Milpitas, CA 95035 USA

[2] Alibaba Cloud, Sunnyvale, CA 94085 USA

来源：

IEEE TRANSACTIONS ON COMMUNICATIONS | 2024年 / 72卷 / 07期

关键词：

Rate-distortion; Noise reduction; Distortion; Neural networks; Training; Electronics packaging; Source coding; Rate-distortion theory; energy-based models; batch denoising; Blahut-Arimoto algorithm;

D O I：

10.1109/TCOMM.2024.3361531

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

In this work, we examine the relationship between rate-distortion theory and energy-based models (EBMs). We demonstrate that EBMs can be used to approximate the rate-distortion approaching posterior, as in the Blahut-Arimoto (BA) algorithm, and to solve batch denoising problems using the posterior distribution learned by EBMs. Our results highlight the potential of EBMs to enhance the efficiency of rate-distortion theory analysis and vice versa.

引用

页码：4072 / 4083

页数：12

共 47 条

[31] Deep Learning Face Attributes in the Wild [J].

Liu, Ziwei ;

Luo, Ping ;

Wang, Xiaogang ;

Tang, Xiaoou .

2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015, :3730-3738

[32]

Metz Luke, 2016, P INT C LEARN REPR

[33] FILTERING RANDOM NOISE FROM DETERMINISTIC SIGNALS VIA DATA-COMPRESSION [J].

NATARAJAN, BK .

IEEE TRANSACTIONS ON SIGNAL PROCESSING, 1995, 43 (11) :2595-2605

[34] Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization [J].

Nguyen, XuanLong ;

Wainwright, Martin J. ;

Jordan, Michael I. .

IEEE TRANSACTIONS ON INFORMATION THEORY, 2010, 56 (11) :5847-5861

[35]

Nowozin S, 2016, ADV NEUR IN, V29

[36]

Paszke A, 2019, ADV NEUR IN, V32

[37] U-Net: Convolutional Networks for Biomedical Image Segmentation [J].

Ronneberger, Olaf ;

Fischer, Philipp ;

Brox, Thomas .

MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION, PT III, 2015, 9351 :234-241

[38]

Sodhi P., 2022, PMLR, P234

[39]

Song J, 2019, P INT C LEARN REPR

[40]

Song Y., 2020, P INT C LEARN REPR

← 1 2 3 4 5 →