Deep learning-based multi-feature semantic segmentation in building extraction from images of UAV photogrammetry

Building information is an essential part of geographic information system (GIS) applications in urban planning and management. However, it changes rapidly with economic growth. Unmanned aerial vehicles (UAV)-based photogrammetry works well in this situation with its advantages of quick and high-resolution data updating. In this paper, in order to improve building extraction accuracy in complex areas where buildings are characterized by various patterns, complex structures, and unique styles, we present a framework which applies deep learning (DL) semantic segmentation to UAV images with digital surface model (DSM) and visible-band difference vegetation index (VDVI). The results show that extraction accuracy improves. The combination of red, green, blue (RGB) and VDVI bands (RGBVI) can effectively distinguish the building area and vegetation. The application of RGB with DSM bands (RGBD) helps separate buildings from ground objects. The combination of RGB, DSM, and VDVI bands (RGBDVI) can identify small buildings which are usually not high and covered partly by tree branches. The proposed method is further applied to an open standard dataset to evaluate its robustness and results indicate an increased overall accuracy from RGB only (93%) to RGBD (97%).