Deep semantic segmentation for visual scene understanding of soil types

One of the state-of-the-art computer vision applications is scene understanding and visual contextual awareness. Despite the numerous detection and classification-based studies, the literature lacks semantic segmentation methods for a more comprehensive and precise understanding of the soil included scene due to the scarcity of annotated datasets; the extracted information from an understood scene is worthwhile in project fleet management, claims management, equipment productivity analysis, safety, and soil classification. Hence, this study presents a vision-based approach for soil-included scene understanding and classifying them into different categories according to ASTM D2488, using semantic segmentation. An annotated dataset of various soil types containing 3043 images was developed to train four Deeplab v3+ variants with modified decoders. Five-fold cross-validation indicates the remarkable performance of the best variant with a mean Jaccard index of 0.9. The application and effects of subpixel upsampling and exit-flow CRF were also examined.