Neural Network Based Multi-Level In-Loop Filtering for Versatile Video Coding

被引：0

作者：

Zhu, Linwei ^{[1
]}

Zhang, Yun ^{[2
]}

Li, Na ^{[1
]}

Wu, Wenhui ^{[3
]}

Wang, Shiqi ^{[4
]}

Kwong, Sam ^{[5
]}

机构：

[1] Chinese Acad Sci, Shenzhen Inst Adv Technol, Shenzhen 518055, Peoples R China

[2] Sun Yat Sen Univ, Sch Elect & Commun Engn, Shenzhen 518107, Peoples R China

[3] Shenzhen Univ, Coll Elect & Informat Engn, Shenzhen 518060, Peoples R China

[4] City Univ Hong Kong, Dept Comp Sci, Hong Kong, Peoples R China

[5] Lingnan Univ, Dept Comp & Decis Sci, Hong Kong, Peoples R China

来源：

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY | 2024年 / 34卷 / 11期

关键词：

In-loop filtering; versatile video coding; neural network; multi-level;

D O I：

10.1109/TCSVT.2024.3420435

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

To further improve the performance of Versatile Video Coding (VVC), a neural network based multi-level in-loop filtering framework for luma and chroma is presented in this letter, which includes Reference pixel Level (RL), Coding tree unit Level (CL), and Frame Level (FL). The neural network based filters in these levels can be flexibly enabled. In RL, the coding performance upper bound is analyzed and asymmetric convolution is designed. In CL, the pixels located at the bottom and rightmost have been assigned greater weights for loss calculation during training. In addition, the co-located luma is adopted in CL and FL chroma filtering for guiding chroma enhancement due to the high correlation between them. For the architecture of neural network, two input channel fusion schemes are combined to enjoy both of their benefits. Extensive experimental results show that the proposed multi-level in-loop filtering method can achieve 6.87%, 32.8%, and 36.9% bit rate reductions on average for Y, U, and V components under all intra configuration, which outperforms the state-of-the-art works.

引用

页码：12092 / 12096

页数：5

共 15 条

[1] Bross B., Chen J., Ohm J.-R., Sullivan G.J., Wang Y.-K., Developments in international video coding standardization after AVC, with an overview of versatile video coding (VVC), Proc. IEEE, 109, 9, pp. 1463-1493, (2021)
[2] Sullivan G.J., Ohm J.-R., Han W.-J., Wiegand T., Overview of the high efficiency video coding (HEVC) standard, IEEE Trans. Circuits Syst. Video Technol., 22, 12, pp. 1649-1668, (2012)
[3] Karczewicz M., Et al., VVC in-loop filters, IEEE Trans. Circuits Syst. Video Technol., 31, 10, pp. 3907-3925, (2021)
[4] Liu C., Sun H., Katto J., Zeng X., Fan Y., QA-filter: A QP-adaptive convolutional neural network filter for video coding, IEEE Trans. Image Process., 31, pp. 3032-3045, (2022)
[5] Ding D., Kong L., Chen G., Liu Z., Fang Y., A switchable deep learning approach for in-loop filtering in video coding, IEEE Trans. Circuits Syst. Video Technol., 30, 7, pp. 1871-1887, (2020)
[6] Huang Z., Sun J., Guo X., Shang M., One-for-all: An efficient variable convolution neural network for in-loop filter of VVC, IEEE Trans. Circuits Syst. Video Technol., 32, 4, pp. 2342-2355, (2022)
[7] Ding Q., Shen L., Yu L., Yang H., Xu M., Patch-wise spatial-temporal quality enhancement for HEVC compressed video, IEEE Trans. Image Process., 30, pp. 6459-6472, (2021)
[8] Huang Z., Sun J., Guo X., Shang M., Adaptive deep reinforcement learning-based in-loop filter for VVC, IEEE Trans. Image Process., 30, pp. 5439-5451, (2021)
[9] Bjontegaard G., Calculation of Average PSNR Differences Between RDCurves, (2001)
[10] Zhu L., Zhang Y., Li N., Jiang G., Kwong S., Deep learningbased intra mode derivation for versatile video coding, ACM Trans. Multimedia Comput., Commun., Appl., 19, 2, pp. 1-20, (2023)

← 1 2 →