Sky Pixel Detection in Outdoor Urban Scenes: U-Net with Transfer Learning

The sky depicts a high visual importance in outdoor scenes, often appearing in video sequences and photos. Sky information is crucial for accurate sky detection in several computer vision applications, such as scene understanding, navigation, surveillance, and weather forecasting. The difficulty of detecting is clarified by variations in the sky's size, weather and lighting conditions, and the sky's reflection on other objects. This article presents a new contribution to address the challenges facing sky detection. A unique dataset was built that includes scenes of distinct lighting and atmospheric phenomena. Additionally, a modified U-Net architecture was proposed with pre-trained models as encoder VGG19, EfficientNetB4, InceptionV3, and DenseNet121 for sky detection to solve outdoor image limitations and evaluate the influence of different encoders when integrated with the U-Net, aiming to identify which encoder describes features of the sky accurately. The proposed approach shows encouraging results; as it presents improved performance over the adjusted U-Net architecture with inceptionv3 on the proposed dataset, achieving mean Intersection over union, dice similarity coefficient, recall, precision, and accuracy of 98.57 %, 99.57 %, 99.41 %, 99.73%, and 99.40 %, respectively. At the same time, the best loss was achieved in U-Net with VGG19 equivalent of 0.09.