Tripartite real-time semantic segmentation network with scene commonality

被引:0
作者
Wang, Chenyang [1 ]
Wang, Chuanxu [1 ]
Liu, Peng [1 ]
Zhang, Zhe [1 ]
Lin, Guocheng [1 ]
机构
[1] Qingdao Univ Sci & Technol, Sch Informat Sci & Technol, Qingdao, Peoples R China
关键词
real-time semantic segmentation; three-branch network; scene commonality; attention mechanism; feature fusion;
D O I
10.1117/1.JEI.33.2.023016
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
The two-branch real-time semantic segmentation network can quickly acquire low-level details and high-level semantics. However, the large contextual gap between them results in adverse impact on their fusion, and limits the further improvement of real-time segmentation accuracy. This paper proposes a tripartite real-time semantic segmentation network with scene commonality (TriSCNet) to address this problem. First, we add a parallel scene commonality branch based on the current two-branch architecture to learn intrinsic common features in similar street scene images, such as the spatial location distribution of various objects and the internal connections between them at the semantic level. Further, with the guidance of commonality, we propose an external branch attention module to enrich and enhance the feature information of traditional two branches. Finally, we utilize an alignment and selective fusion module to correct the misaligned context in the semantic branch and highlight the essential spatial information in the detailed branch. Our proposed TriSCNet achieves an excellent trade-off between accuracy and speed, yielding 77.9% mIOU at 67.2 FPS on Cityscapes test set and 75.8% mIOU at 127.4 FPS on CamVid test set, respectively. (c) 2024 SPIE and IS&T
引用
收藏
页数:13
相关论文
共 36 条
[1]   SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation [J].
Badrinarayanan, Vijay ;
Kendall, Alex ;
Cipolla, Roberto .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2017, 39 (12) :2481-2495
[2]   Semantic object classes in video: A high-definition ground truth database [J].
Brostow, Gabriel J. ;
Fauqueur, Julien ;
Cipolla, Roberto .
PATTERN RECOGNITION LETTERS, 2009, 30 (02) :88-97
[3]  
Chaurasia A, 2017, 2017 IEEE VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP)
[4]   Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation [J].
Chen, Liang-Chieh ;
Zhu, Yukun ;
Papandreou, George ;
Schroff, Florian ;
Adam, Hartwig .
COMPUTER VISION - ECCV 2018, PT VII, 2018, 11211 :833-851
[5]   The Cityscapes Dataset for Semantic Urban Scene Understanding [J].
Cordts, Marius ;
Omran, Mohamed ;
Ramos, Sebastian ;
Rehfeld, Timo ;
Enzweiler, Markus ;
Benenson, Rodrigo ;
Franke, Uwe ;
Roth, Stefan ;
Schiele, Bernt .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :3213-3223
[6]   Rethinking BiSeNet For Real-time Semantic Segmentation [J].
Fan, Mingyuan ;
Lai, Shenqi ;
Huang, Junshi ;
Wei, Xiaoming ;
Chai, Zhenhua ;
Luo, Junfeng ;
Wei, Xiaolin .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :9711-9720
[7]   Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges [J].
Feng, Di ;
Haase-Schutz, Christian ;
Rosenbaum, Lars ;
Hertlein, Heinz ;
Glaser, Claudius ;
Timm, Fabian ;
Wiesbeck, Werner ;
Dietmayer, Klaus .
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2021, 22 (03) :1341-1360
[8]   Dual Attention Network for Scene Segmentation [J].
Fu, Jun ;
Liu, Jing ;
Tian, Haijie ;
Li, Yong ;
Bao, Yongjun ;
Fang, Zhiwei ;
Lu, Hanqing .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :3141-3149
[9]   FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation [J].
Gao, Guangwei ;
Xu, Guoan ;
Li, Juncheng ;
Yu, Yi ;
Lu, Huimin ;
Yang, Jian .
IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 :3273-3283
[10]  
Hong YD, 2021, Arxiv, DOI arXiv:2101.06085