DFAMNet: dual fusion attention multi-modal network for semantic segmentation on LiDAR point clouds

Semantic segmentation of outdoor point clouds is an important task in the field of computer vision, aiming to classify outdoor point cloud data into different semantic categories. The methods based on pure point cloud have some shortcomings, such as incomplete information and difficulty in processing incomplete data. In the paper, it proposes pseudo point cloud method to align image with point cloud. The image features are extracted through a 2D network, and then the point cloud is mapped onto the image to obtain the corresponding pixel features, forming the pseudo point cloud. Then the dual fusion attention mechanism is designed to fuse the features of point cloud and pseudo point cloud. It improves the efficiency of the fusion network. The experimental results show that this method outperforms existing methods on the large-scale SemanticKITTI benchmark and achieves third place performance on the NuScenes benchmark. Code is available at https://github.com/Pdsn5/DFAMNet.