SDRTV-to-HDRTV via Hierarchical Dynamic Context Feature Mapping

被引:20
作者
He, Gang [1 ,2 ]
Xu, Kepeng [1 ]
Xu, Li [1 ]
Wu, Chang [1 ]
Sun, Ming [2 ]
Wen, Xing [2 ]
Tai, Yu-Wing [2 ]
机构
[1] Xidian Univ, Xian, Peoples R China
[2] Kuaishou Technol, Beijing, Peoples R China
来源
PROCEEDINGS OF THE 30TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2022 | 2022年
关键词
Standard Dynamic Range; High Dynamic Range; Feature Transformation; Dynamic Convolution; Neural Network;
D O I
10.1145/3503161.3548043
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
In this work, we address the task of SDR videos to HDR videos(SDRTV-to-HDRTV conversion). Previous approaches use global feature modulation for SDRTV-to-HDRTV conversion. Feature modulation scales and shifts the features in the original feature space, which has limited mapping capability. In addition, the global image mapping cannot restore detail in HDR frames due to the luminance differences in different regions of SDR frames. To resolve the appeal, we propose a two-stage solution. The first stage is a hierarchical Dynamic Context feature mapping (HDCFM) model. HDCFM learns the SDR frame to HDR frame mapping function via hierarchical feature modulation (HME and HM) module and a dynamic context feature transformation (DYCT) module. The HME estimates the feature modulation vector, HM is capable of hierarchical feature modulation, consisting of global feature modulation in series with local feature modulation, and is capable of adaptive mapping of local image features. The DYCT module constructs a feature transformation module in conjunction with the context, which is capable of adaptively generating a feature transformation matrix for feature mapping. Compared with simple feature scaling and shifting, the DYCT module can map features into a new feature space and thus has a more excellent feature mapping capability. In the second stage, we introduce a patch discriminator-based context generation model PDCG to obtain subjective quality enhancement of over-exposed regions. The proposed method can achieve state-of-the-art objective and subjective quality results. Specifically, HDCFM achieves a PSNR gain of 0.81 dB at about 100K parameters. The number of parameters is 1/14th of the previous state-of-the-art methods. The test code will be released on https://github.com/cooperlike/HDCFM.
引用
收藏
页码:2890 / 2898
页数:9
相关论文
共 47 条
[41]   Overview of the High Efficiency Video Coding (HEVC) Standard [J].
Sullivan, Gary J. ;
Ohm, Jens-Rainer ;
Han, Woo-Jin ;
Wiegand, Thomas .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2012, 22 (12) :1649-1668
[42]   Deepfakes and beyond: A Survey of face manipulation and fake detection [J].
Tolosana, Ruben ;
Vera-Rodriguez, Ruben ;
Fierrez, Julian ;
Morales, Aythami ;
Ortega-Garcia, Javier .
INFORMATION FUSION, 2020, 64 :131-148
[43]  
Wang SY, 2020, CHIN CONTR CONF, P6954, DOI 10.23919/CCC50068.2020.9189580
[44]   Deep HDR Imaging via A Non-Local Network [J].
Yan, Qingsen ;
Zhang, Lei ;
Liu, Yu ;
Zhu, Yu ;
Sun, Jinqiu ;
Shi, Qinfeng ;
Zhang, Yanning .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2020, 29 :4308-4322
[45]  
Zeng Hui, 2020, IEEE T PATTERN ANAL, V2020
[46]  
Zhou Jingkai, 2021, ARXIV210414107CSCV
[47]   Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks [J].
Zhu, Jun-Yan ;
Park, Taesung ;
Isola, Phillip ;
Efros, Alexei A. .
2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, :2242-2251