A layer-2 solution for inspecting large-scale photovoltaic arrays through aerial LWIR multiview photogrammetry and deep learning: A hybrid data-centric and model-centric approach

Defective components within solar photovoltaic (PV) arrays overheat, resulting in particular temperature patterns under the long-wave thermal infrared (LWIR) spectrum. The detection and on-field localization of these patterns is of paramount aid to the operations and maintenance of PV installations. In this context, we develop a two-layer end-to-end inspection solution for the detection, quantification and on-field localization of overheated regions on PV arrays from LWIR UAV imagery. Layer 1 generates a georeferenced orthomosaic of the inspected site via a Structure from Motion-MultiView Stereo (SfM-MVS) photogrammetric acquisition/post-processing workflow. Layer 2 is a tile-based deep semantic segmentation stage that extracts and quantifies the affected regions from the generated orthomosaic. We collect aerial images from 103 PV sites, comprising approximately 342 000 modules. After a SfM-MVS workflow, we produce and annotate 7910 orthorectified unique affected tiles, posteriorly augmented to prepare the state-of-the-art dataset in terms of size and representativeness. Through a training/cross-validation and test process, we investigate the implementation of 9 models in the segmentation process: FCN, U-Net, FPN, DeepLab, LinkNet, DANet, CFNet, ACFNet and TransU-Net, each of which experimented with 2 backbones: ResNet50 and DenseNet121. The models feature efficient encoder-to-decoder feature map transfers, pyramidal feature recognition, spatial and channel attention, feature co-occurrence, class center as well as vision transformers. The best performance is achieved by FPN-DenseNet121, with a mean mIoU of 93.44% and an F1-score of 96.39% on our test set. The two-layer solution takes the best of the data-centric and model-centric paradigms, alongside addressing the limitations of conventional inspection procedures. It is put into a concrete application framework, where it provides a pixel-based and a tile-based quantification of the affected regions within a PV plant. The results are promising, and the selected model can be deployed efficiently for extensive aerial monitoring of large-scale PV plants.