Learned Rate Control for Frame-Level Adaptive Neural Video Compression via Dynamic Neural Network

被引:0
作者
Zhang, Chenhao [1 ]
Gao, Wei [1 ,2 ]
机构
[1] Peking Univ, Shenzhen Grad Sch, SECE, Shenzhen, Peoples R China
[2] Peng Cheng Lab, Shenzhen, Peoples R China
来源
COMPUTER VISION - ECCV 2024, PT LXXXV | 2025年 / 15143卷
关键词
Neural Video Compression; Rate Control; Rate-Distortion-Complexity Optimization;
D O I
10.1007/978-3-031-73013-9_14
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Neural Video Compression (NVC) has achieved remarkable performance in recent years. However, precise rate control remains a challenge due to the inherent limitations of learning-based codecs. To solve this issue, we propose a dynamic video compression framework designed for variable bitrate scenarios. First, to achieve variable bitrate implementation, we propose the Dynamic-Route Autoencoder with variable coding routes, each occupying partial computational complexity of the whole network and navigating to a distinct RD trade-off. Second, to approach the target bitrate, the Rate Control Agent estimates the bitrate of each route and adjusts the coding route of DRA at run time. To encompass a broad spectrum of variable bitrates while preserving overall RD performance, we employ the Joint-Routes Optimization strategy, achieving collaborative training of various routes. Extensive experiments on the HEVC and UVG datasets show that the proposed method achieves an average BD-Rate reduction of 14.8% and BD-PSNR gain of 0.47 dB over state-of-the-art methods while maintaining an average bitrate error of 1.66%, achieving Rate-Distortion-Complexity Optimization (RDCO) for various bitrate and bitrate-constrained applications.
引用
收藏
页码:239 / 255
页数:17
相关论文
共 38 条
[31]   A Computationally Efficient Neural Video Compression Accelerator Based on a Sparse CNN-Transformer Hybrid Network [J].
Zhang, Siyu ;
Mao, Wendong ;
Shi, Huihong ;
Wang, Zhongfeng .
2024 DESIGN, AUTOMATION & TEST IN EUROPE CONFERENCE & EXHIBITION, DATE, 2024,
[32]   Frame level rate control algorithm based on GOP level quality dependency for low-delay hierarchical video coding [J].
Zhang, Meng ;
Zhou, Wei ;
Wei, Henglu ;
Zhou, Xin ;
Duan, Zhemin .
SIGNAL PROCESSING-IMAGE COMMUNICATION, 2020, 88
[33]   View synthesis distortion model based frame level rate control optimization for multiview depth video coding [J].
Wang, Xu ;
Kwong, Sam ;
Yuan, Hui ;
Zhang, Yun ;
Pan, Zhaoqing .
SIGNAL PROCESSING, 2015, 112 :189-198
[34]   A Convolutional Neural Network-Based Approach to Rate Control in HEVC Intra Coding [J].
Li, Ye ;
Li, Bin ;
Liu, Dong ;
Chen, Zhibo .
2017 IEEE VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2017,
[35]   A convolutional neural network-based rate control algorithm for VVC intra coding [J].
Wang, Jiafeng ;
Shang, Xiwu ;
Zhao, Xiaoli ;
Zhang, Yuhuai .
DISPLAYS, 2024, 82
[36]   A bandwidth adaptive rate control scheme of MPEG transcoder for video streaming over non QOS-guaranteed network [J].
Wakui, M ;
Nagayoshi, I ;
Tominaga, H ;
Hanamura, T .
7TH WORLD MULTICONFERENCE ON SYSTEMICS, CYBERNETICS AND INFORMATICS, VOL, III, PROCEEDINGS: COMMUNICATION, NETWORK AND CONTROL SYSTEMS, TECHNOLOGIES AND APPLICATIONS, 2003, :24-29
[37]   Network-Adaptive Rate Control and Unequal Loss Protection with TCP-Friendly Protocol for Scalable Video Over Internet [J].
Qian Zhang ;
Wenwu Zhu ;
Ya-Qin Zhang .
Journal of VLSI signal processing systems for signal, image and video technology, 2003, 34 :67-81
[38]   Network-adaptive rate control and unequal loss protection with TCP-friendly protocol for scalable video over Internet [J].
Zhang, Q ;
Zhu, WW ;
Zhang, YQ .
JOURNAL OF VLSI SIGNAL PROCESSING SYSTEMS FOR SIGNAL IMAGE AND VIDEO TECHNOLOGY, 2003, 34 (1-2) :67-81