LVE-S2D: Low-Light Video Enhancement From Static to Dynamic

被引:45
作者
Peng, Bo [1 ]
Zhang, Xuanyu [1 ]
Lei, Jianjun [1 ]
Zhang, Zhe [1 ]
Ling, Nam [2 ]
Huang, Qingming [3 ]
机构
[1] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[2] Santa Clara Univ, Dept Comp Sci & Engn, Santa Clara, CA 95053 USA
[3] Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China
基金
中国国家自然科学基金;
关键词
Image enhancement; Video sequences; Training; Task analysis; Histograms; Correlation; Lighting; Low-light video enhancement; sliding window; temporal correlation; deep learning; IMAGE; FRAMEWORK; RETINEX;
D O I
10.1109/TCSVT.2022.3190916
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Recently, deep-learning-based low-light video enhancement methods have drawn wide attention and achieved remarkable performance. However, limited by the difficulty in collecting dynamic low-light and well-lighted video pairs in real scenes, how to construct video sequences for supervised learning and design a low-light enhancement network for real dynamic video remains a challenge. In this paper, we propose a simple yet effective low-light video enhancement method (LVE-S2D), which generates dynamic video training pairs from static videos, and enhances the low-light video by mining dynamic temporal information. To obtain low-light and well-lighted video pairs, a sliding window-based dynamic video generation mechanism is designed to produce pseudo videos with rich dynamic temporal information. Then, a siamese dynamic low-light video enhancement network is presented, which effectively utilizes temporal correlation between adjacent frames to enhance the video frames. Extensive experimental results demonstrate that the proposed method not only achieves superior performance on static low-light videos, but also outperforms the state-of-the-art methods on real dynamic low-light videos.
引用
收藏
页码:8342 / 8352
页数:11
相关论文
共 49 条
[1]  
Abadi M, 2016, PROCEEDINGS OF OSDI'16: 12TH USENIX SYMPOSIUM ON OPERATING SYSTEMS DESIGN AND IMPLEMENTATION, P265
[2]   A Histogram Modification Framework and Its Application for Image Contrast Enhancement [J].
Arici, Tarik ;
Dikbas, Salih ;
Altunbasak, Yucel .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2009, 18 (09) :1921-1935
[3]   Describing Video With Attention-Based Bidirectional LSTM [J].
Bin, Yi ;
Yang, Yang ;
Shen, Fumin ;
Xie, Ning ;
Shen, Heng Tao ;
Li, Xuelong .
IEEE TRANSACTIONS ON CYBERNETICS, 2019, 49 (07) :2631-2641
[4]   Seeing Motion in the Dark [J].
Chen, Chen ;
Chen, Qifeng ;
Do, Minh N. ;
Koltun, Vladlen .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :3184-3193
[5]   Learning to See in the Dark [J].
Chen, Chen ;
Chen, Qifeng ;
Xu, Jia ;
Koltun, Vladlen .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :3291-3300
[6]  
Dong X, 2011, IEEE INT CON MULTI
[7]   Source Distortion Temporal Propagation Analysis for Random-Access Hierarchical Video Coding Optimization [J].
Gao, Yanbo ;
Zhu, Ce ;
Li, Shuai ;
Yang, Tianwu .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2019, 29 (02) :546-559
[8]   Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement [J].
Guo, Chunle ;
Li, Chongyi ;
Guo, Jichang ;
Loy, Chen Change ;
Hou, Junhui ;
Kwong, Sam ;
Cong, Runmin .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :1777-1786
[9]   LIME: Low-Light Image Enhancement via Illumination Map Estimation [J].
Guo, Xiaojie ;
Li, Yu ;
Ling, Haibin .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2017, 26 (02) :982-993
[10]   Learning to See Moving Objects in the Dark [J].
Jiang, Haiyang ;
Zheng, Yinqiang .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :7323-7332