Improving Semantic Image Segmentation via Label Fusion in Semantically Textured Meshes

被引:0
|
作者
Fervers, Florian [1 ]
Breuer, Timo [1 ]
Stachowiak, Gregor [1 ]
Bullinger, Sebastian [1 ]
Bodensteiner, Christoph [1 ]
Arens, Michael [1 ]
机构
[1] Fraunhofer IOSB, D-76275 Ettlingen, Germany
来源
PROCEEDINGS OF THE 17TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS (VISAPP), VOL 5 | 2022年
关键词
Semantic Segmentation; Mesh Reconstruction; Label Fusion;
D O I
10.5220/0010841800003124
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Models for semantic segmentation require a large amount of hand-labeled training data which is costly and time-consuming to produce. For this purpose, we present a label fusion framework that is capable of improving semantic pixel labels of video sequences in an unsupervised manner. We make use of a 3D mesh representation of the environment and fuse the predictions of different frames into a consistent representation using semantic mesh textures. Rendering the semantic mesh using the original intrinsic and extrinsic camera parameters yields a set of improved semantic segmentation images. Due to our optimized CUDA implementation, we are able to exploit the entire c-dimensional probability distribution of annotations over c classes in an uncertainty-aware manner. We evaluate our method on the Scannet dataset where we improve annotations produced by the state-of-the-art segmentation network ESANet from 52.05% to 58.25% pixel accuracy. We publish the source code of our framework online to foster future research in this area (https://github.com/fferflo/semantic-meshes). To the best of our knowledge, this is the first publicly available label fusion framework for semantic image segmentation based on meshes with semantic textures.
引用
收藏
页码:509 / 516
页数:8
相关论文
共 50 条
  • [41] MFAFNet: A Multiscale Fully Attention Fusion Network for Remote Sensing Image Semantic Segmentation
    Dang, Yuanyuan
    Gao, Yu
    Liu, Bing
    IEEE ACCESS, 2024, 12 : 123388 - 123400
  • [42] Weakly Supervised Remote Sensing Image Semantic Segmentation With Pseudo-Label Noise Suppression
    Lu, Xiao
    Jiang, Zhiguo
    Zhang, Haopeng
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62
  • [43] Remote Sensing Image Coding for Machines on Semantic Segmentation via Contrastive Learning
    Zhang, Junxi
    Chen, Zhenzhong
    Liu, Shan
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62
  • [44] Thermal image-guided complementary masking with multiscale fusion for multi-spectral image semantic segmentation
    Chen, Zeyang
    Hu, Mingnan
    Chen, Bo
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2025, 150
  • [45] Semantic segmentation of mobile mapping point clouds via multi-view label transfer
    Peters, Torben
    Brenner, Claus
    Schindler, Konrad
    ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2023, 202 : 30 - 39
  • [46] SRCBTFusion-Net: An Efficient Fusion Architecture via Stacked Residual Convolution Blocks and Transformer for Remote Sensing Image Semantic Segmentation
    Chen, Junsong
    Yi, Jizheng
    Chen, Aibin
    Lin, Hui
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61 : 1 - 16
  • [47] CMTFNet: CNN and Multiscale Transformer Fusion Network for Remote-Sensing Image Semantic Segmentation
    Wu, Honglin
    Huang, Peng
    Zhang, Min
    Tang, Wenlong
    Yu, Xinyu
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61
  • [48] INVITATION: A Framework for Enhancing UAV Image Semantic Segmentation Accuracy Through Depth Information Fusion
    Zhang, Xiaodong
    Zhou, Wenlin
    Chen, Guanzhou
    Wang, Jiaqi
    Yang, Qingyuan
    Tan, Xiaoliang
    Wang, Tong
    Chen, Yifei
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2025, 22
  • [49] Drivable Dirt Road Region Identification Using Image and Point Cloud Semantic Segmentation Fusion
    Park, Jihun
    Yoo, Hyunsung
    Wang, Yooseung
    IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2022, 23 (08) : 13203 - 13216
  • [50] Research on Image Semantic Segmentation Based on Hybrid Cascade Feature Fusion and Detailed Attention Mechanism
    Du, Zuoqiang
    Liang, Yuan
    IEEE ACCESS, 2024, 12 : 62365 - 62377