Generalizing event-based HDR imaging to various exposures

被引:1
作者
Li, Xiaopeng [1 ]
Lu, Qingyang [2 ]
Fan, Cien [1 ]
Zhao, Chen [1 ]
Zou, Lian [1 ]
Yu, Lei [1 ]
机构
[1] Wuhan Univ, Sch Elect Informat, Wuhan 430072, Peoples R China
[2] Univ Calgary, Dept Comp Sci, Calgary, AB T2N 1N4, Canada
基金
中国国家自然科学基金;
关键词
High dynamic range imaging; Event-based vision; Multi-modal fusion; Various exposures; NETWORK; RECONSTRUCTION; FUSION; TRACKING;
D O I
10.1016/j.neucom.2024.128132
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Single-exposure High Dynamic Range Imaging (HDRI), as a typical ill-posed problem, has attracted extensive attention from researchers. However, restoration in real-world scenarios has always been an intractable task due to various exposures and noise artifacts. This work proposes an event-based HDRI framework that generalizes to scenes under various exposures by exploiting the high dynamic range of events. To address the challenge of processing diverse exposures, we propose an exposure-aware network incorporating the exposure attention fusion module, which facilitates the adaptive fusion of SDR image and event features. Moreover, the problem of noise in extremely under-exposed regions and events is effectively alleviated by introducing a self-supervised loss, namely EDDN, which effectively enhances the details of saturated areas while simultaneously decreasing noise. We conduct novel event-based HDRI datasets to evaluate our proposed method for benchmarking with diverse exposed images. Comprehensive experiments have demonstrated that our method outperforms the state-of-the-art.
引用
收藏
页数:12
相关论文
共 79 条
  • [11] EventZoom: Learning to Denoise and Super Resolve Neuromorphic Events
    Duan, Peiqi
    Wang, Zihao W.
    Zhou, Xinyu
    Ma, Yi
    Shi, Boxin
    [J]. 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 12819 - 12828
  • [12] HDR image reconstruction from a single exposure using deep CNNs
    Eilertsen, Gabriel
    Kronander, Joel
    Denes, Gyorgy
    Mantiuk, Rafal K.
    Unger, Jonas
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2017, 36 (06):
  • [13] Deep Reverse Tone Mapping
    Endo, Yuki
    Kanamori, Yoshihiro
    Mitani, Jun
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2017, 36 (06):
  • [14] Event-Based Vision: A Survey
    Gallego, Guillermo
    Delbruck, Tobi
    Orchard, Garrick Michael
    Bartolozzi, Chiara
    Taba, Brian
    Censi, Andrea
    Leutenegger, Stefan
    Davison, Andrew
    Conradt, Jorg
    Daniilidis, Kostas
    Scaramuzza, Davide
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (01) : 154 - 180
  • [15] TMMF: Temporal Multi-Modal Fusion for Single-Stage Continuous Gesture Recognition
    Gammulle, Harshala
    Denman, Simon
    Sridharan, Sridha
    Fookes, Clinton
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 : 7689 - 7701
  • [16] SPADE-E2VID: Spatially-Adaptive Denormalization for Event-Based Video Reconstruction
    Gantier Cadena, Pablo Rodrigo
    Qian, Yeqiang
    Wang, Chunxiang
    Yang, Ming
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 : 2488 - 2500
  • [17] Gouveia LC, 2014, IEEE IMTC P, P1098, DOI 10.1109/I2MTC.2014.6860912
  • [18] Guo C., 2022, P AS C COMP VIS MAC, P3155
  • [19] Hybrid High Dynamic Range Imaging fusing Neuromorphic and Conventional Images
    Han, Jin
    Yang, Yixin
    Duan, Peiqi
    Zhou, Chu
    Ma, Lei
    Xu, Chao
    Huang, Tiejun
    Sato, Imari
    Shi, Boxin
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2023, 45 (07) : 8553 - 8565
  • [20] Neuromorphic Camera Guided High Dynamic Range Imaging
    Han, Jin
    Zhou, Chu
    Duan, Peiqi
    Tang, Yehui
    Xu, Chang
    Xu, Chao
    Huang, Tiejun
    Shi, Boxin
    [J]. 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, : 1727 - 1736