PASIFTNet: Scale-and-Directional-Aware Semantic Segmentation of Point Clouds

Point clouds obey the sparsity, disorderliness and irregularity properties, leading to noisy or unrobust features during the 3D semantic segmentation task. Existing approaches cannot fully mine local geom-etry and context information of point clouds, due to their irrational feature learning or neighborhood selection schemes. In this paper, we propose a Point-Atrous SIFT Network (PASIFTNet) for learning multi-scale multi-directional features of point clouds. PASIFTNet is a hierarchical encoder-decoder net-work, which combines the Point-Atrous SIFT (PASIFT) modules and edge-preserved pooling/unpooling modules alternatively during the encoder/decoder stage. The key component of PASIFTNet is the Point-Atrous Orientation Encoding unit of the PASIFT module, which can arbitrarily expand its receptive fields to incorporate larger context information and extract scale-and-directional-aware feature point information, benefiting from the quadrant-wise SIFT-like point-atrous convolution. Moreover, the edge -preserved pooling/unpooling modules complement PASIFTNet by preserving the edge features and recovering the high-dimensional features of point clouds. We conduct experiments on two public 3D point cloud datasets: ScanNet, S3DIS and a real-world unlabeled dataset FARO-3 collected by the FARO laser scanner. The quantitative results show that, PASIFTNet achieves 86.8% overall accuracy on ScanNet and achieves 86.5% overall accuracy and 68.3% mean intersection-over-union on S3DIS. Moreover, PASIFTNet exhibits a satisfactory robustness and generalization ability towards unknown scenes on FARO-3.& COPY; 2022 Elsevier Ltd. All rights reserved.