Deep Generative Modeling Reshapes Compression and Transmission: From Efficiency to Resiliency

被引:3
作者
Dai, Jincheng [1 ]
Qin, Xiaoqi [1 ]
Wang, Sixian [1 ]
Xu, Lexi [2 ]
Niu, Kai [1 ]
Zhang, Ping [1 ]
机构
[1] Beijing Univ Posts & Telecommun, Key Lab Universal Wireless Commun, Minist Educ, Beijing, Peoples R China
[2] Beijing Univ Posts & Telecommun, Sch Informat & Commun Engn, Beijing, Peoples R China
基金
中国国家自然科学基金; 北京市自然科学基金;
关键词
Data compression - Image coding - Learning systems - Semantics;
D O I
10.1109/MWC.005.2300574
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Information theory and machine learning are inextricably linked and have even been referred to as "two sides of the same coin." One particularly elegant connection is the essential equivalence between probabilistic generative modeling and data compression or transmission. In this article, we reveal the dual-functionality of deep generative models that reshapes both data compression for efficiency and transmission error concealment for resiliency. We present how the contextual predictive capabilities of powerful generative models can be well positioned to be strong compressors and estimators. In this sense, we advocate for viewing the deep generative modeling problem through the lens of end-to-end communications, and evaluating the compression and error restoration capabilities of foundation generative models. We show that the kernel of many large generative models is a powerful predictor that can capture complex relationships among semantic latent variables, and the communication viewpoints provide novel insights into semantic feature tokenization, contextual learning, and usage of deep generative models. In summary, our article highlights the essential connections of generative AI to source and channel coding techniques, and motivates researchers to make further explorations in this emerging topic.
引用
收藏
页码:48 / 56
页数:9
相关论文
共 15 条
[1]   Multi-Realism Image Compression with a Conditional Generator [J].
Agustsson, Eirikur ;
Minnen, David ;
Toderici, George ;
Mentzer, Fabian .
2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2023, :22324-22333
[2]   Nonlinear Transform Coding [J].
Balle, Johannes ;
Chou, Philip A. ;
Minnen, David ;
Singh, Saurabh ;
Johnston, Nick ;
Agustsson, Eirikur ;
Hwang, Sung Jin ;
Toderici, George .
IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 2021, 15 (02) :339-353
[3]  
Blau Y, 2019, PR MACH LEARN RES, V97
[4]   Deep Joint Source-Channel Coding for Wireless Image Transmission [J].
Bourtsoulatze, Eirina ;
Kurka, David Burth ;
Gunduz, Deniz .
IEEE TRANSACTIONS ON COGNITIVE COMMUNICATIONS AND NETWORKING, 2019, 5 (03) :567-579
[5]   MaskGIT: Masked Generative Image Transformer [J].
Chang, Huiwen ;
Zhang, Han ;
Jiang, Lu ;
Liu, Ce ;
Freeman, William T. .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, :11305-11315
[6]   Nonlinear Transform Source-Channel Coding for Semantic Communications [J].
Dai, Jincheng ;
Wang, Sixian ;
Tan, Kailin ;
Si, Zhongwei ;
Qin, Xiaoqi ;
Niu, Kai ;
Zhang, Ping .
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2022, 40 (08) :2300-2316
[7]  
Delétang G, 2024, Arxiv, DOI [arXiv:2309.10668, 10.48550/arXiv.2309.10668]
[8]  
El-Nouby A, 2023, Arxiv, DOI [arXiv:2212.07372, 10.48550/ARXIV.2212.07372, DOI 10.48550/ARXIV.2212.07372]
[9]   Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications [J].
Gunduz, Deniz ;
Qin, Zhijin ;
Aguerri, Inaki Estella ;
Dhillon, Harpreet S. ;
Yang, Zhaohui ;
Yener, Aylin ;
Wong, Kai Kit ;
Chae, Chan-Byoung .
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2023, 41 (01) :5-41
[10]  
Li CY, 2024, Arxiv, DOI arXiv:2402.16749