DDFL: Dual-Domain Feature Learning for nighttime semantic segmentation

被引:2
作者
Lin, Xiao [1 ,2 ,3 ]
Tan, Peiwen [1 ]
Wang, Zhengkai [1 ]
Ma, Lizhuang [4 ,5 ]
Li, Yan [1 ]
机构
[1] Shanghai Normal Univ, Artificial Intelligence Educ Res Inst, Coll Informat Mech & Elect Engn, Shanghai 200234, Peoples R China
[2] Shanghai Normal Univ, Shanghai Engn Res Ctr Intelligent Educ & Big Data, Shanghai 200234, Peoples R China
[3] Res Base Online Educ Shanghai Middle & Primary Sch, Shanghai 200234, Peoples R China
[4] East China Normal Univ, Coll Comp Sci & Technol, Shanghai 200062, Peoples R China
[5] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
基金
中国国家自然科学基金;
关键词
Semantic segmentation; Exposure correction; Frequency domain features; Dual domain fusion;
D O I
10.1016/j.displa.2024.102685
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Nighttime semantic segmentation has been playing a critical role in intelligent transportation, building safety and urban management. However, nighttime scenes present some challenges such as complex structures, multiple light sources, uneven lighting and blurry image noise, which severely degrade the segmentation quality of nighttime images. To address these challenges, we propose a Dual-Domain Feature Learning (DDFL) model for nighttime semantic segmentation. Our approach introduces three innovative ideas. First, we establish an exposure correction module to address the impact of lighting differences on the model's learning, so as to maximally restore the pixel distortion and blurry areas caused by artificial light in nighttime scenes. Second, we incorporate frequency domain information into the nighttime segmentation task to give the model stronger discrimination ability. Finally, we introduce a dual-domain fusion module to complement the information of learning from the spatial and frequency domains in a cross -fusion manner, enabling the network to perceive semantic information while preserving details. The proposed model was experimentally tested on the Nightcity, Nightcity+ and BDD100k datasets. Our results demonstrate that our model outperforms mainstream models, achieving mIoU scores of 56.73%, 57.41% and 28.97%, respectively, under different lighting, image exposure levels, and resolutions. These results show that our model is capable of segmenting nighttime scenes efficiently in a high-quality way.
引用
收藏
页数:12
相关论文
共 56 条
[1]   Learning Multi-Scale Photo Exposure Correction [J].
Afifi, Mahmoud ;
Derpanis, Konstantinos G. ;
Ommer, Bjoern ;
Brown, Michael S. .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :9153-9163
[2]   Quadratic polynomial guided fuzzy C-means and dual attention mechanism for medical image segmentation [J].
Cai, Weiwei ;
Zhai, Bo ;
Liu, Yun ;
Liu, Runmin ;
Ning, Xin .
DISPLAYS, 2021, 70
[3]   APPLICATION OF FOURIER ANALYSIS TO VISIBILITY OF GRATINGS [J].
CAMPBELL, FW ;
ROBSON, JG .
JOURNAL OF PHYSIOLOGY-LONDON, 1968, 197 (03) :551-&
[4]   Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation [J].
Chen, Liang-Chieh ;
Zhu, Yukun ;
Papandreou, George ;
Schroff, Florian ;
Adam, Hartwig .
COMPUTER VISION - ECCV 2018, PT VII, 2018, 11211 :833-851
[5]   DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs [J].
Chen, Liang-Chieh ;
Papandreou, George ;
Kokkinos, Iasonas ;
Murphy, Kevin ;
Yuille, Alan L. .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2018, 40 (04) :834-848
[6]  
Chi L., 2020, P 34 INT C NEUR INF, V33, P4479, DOI DOI 10.5555/3495724.3496100
[7]   The Cityscapes Dataset for Semantic Urban Scene Understanding [J].
Cordts, Marius ;
Omran, Mohamed ;
Ramos, Sebastian ;
Rehfeld, Timo ;
Enzweiler, Markus ;
Benenson, Rodrigo ;
Franke, Uwe ;
Roth, Stefan ;
Schiele, Bernt .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :3213-3223
[8]   NightLab: A Dual-level Architecture with Hardness Detection for Segmentation at Night [J].
Deng, Xueqing ;
Wang, Peng ;
Lian, Xiaochen ;
Newsam, Shawn .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :16917-16927
[9]  
Dong S., 2023, IEEE Trans. Intell. Veh.
[10]   GEBNet: Graph-Enhancement Branch Network for RGB-T Scene Parsing [J].
Dong, Shaohua ;
Zhou, Wujie ;
Qian, Xiaohong ;
Yu, Lu .
IEEE SIGNAL PROCESSING LETTERS, 2022, 29 :2273-2277