Fruit Tree Canopy Segmentation by Unmanned Aerial Vehicle Photogrammetry Coupled on Convolutional Neural Network and Attention Mechanism

The segmentation of fruit tree canopy based on Unmanned Aerial Vehicle (UAV) visible spectral images is greatly influenced by complex background information such as topographic relief, shrubs, and weeds. Although existing deep neural networks can improve the robustness of canopy segmentation to a certain extent, they ignore the global context and local detailed information of the canopy due to limited receptive field and information interaction, which restricts the improvement of canopy segmentation accuracy. To address these issues, this paper introduces the Canopy Height Model (CHM) and deep learning algorithms, and proposes a fruit tree canopy segmentation method that couples Convolutional Neural Networks (CNN) and Attention Mechanisms (AM) based on UAV photogrammetry. This method first constructs a coupled deep neural network based on CNN and AM through transfer learning to extract both the local and global high-level contextual features of fruit tree canopies. Meanwhile, considering the correlation between deep semantic features and the position information of fruit tree canopies, a local and global feature fusion module is designed to achieve collaborative tree canopy segmentation of attributes and spatial positions. Taking the citrus tree canopy segmentation as an example, the experimental results demonstrate that the use of the CHM can effectively suppress the influence of topographic relief. Our proposed method can also significantly reduce the interference of underlying weeds or shrubs on canopy segmentation, and achieves the highest Overall Accuracy (OA), F1 score, and mean Intersection over Union (mIoU) of 97.57%, 95.49%, and 94.05%, respectively. Compared with other state-of-the-art networks such as SegFormer, SETR_PUP,TransUNet, TransFuse, and CCTNet, the mIoU obtained by the proposed method increases by 1.79%, 8.83%, 1.16%, 1.43%, and 1.85%, respectively. The proposed method can achieve high-precision segmentation of fruit tree canopies with complex background information, which has important practical value for understanding the growth status of fruit trees and fine management of orchards. © 2023 Chinese Society of Agricultural Engineering. All rights reserved.