Automatic segmentation of stem and leaf components and individual maize plants in field terrestrial LiDAR data using convolutional neural networks

High-throughput maize phenotyping at both organ and plant levels plays a key role in molecular breed-ing for increasing crop yields. Although the rapid development of light detection and ranging (LiDAR) pro-vides a new way to characterize three-dimensional (3D) plant structure, there is a need to develop robust algorithms for extracting 3D phenotypic traits from LiDAR data to assist in gene identification and selec-tion. Accurate 3D phenotyping in field environments remains challenging, owing to difficulties in seg-mentation of organs and individual plants in field terrestrial LiDAR data. We describe a two-stage method that combines both convolutional neural networks (CNNs) and morphological characteristics to segment stems and leaves of individual maize plants in field environments. It initially extracts stem points using the PointCNN model and obtains stem instances by fitting 3D cylinders to the points. It then segments the field LiDAR point cloud into individual plants using local point densities and 3D morpho-logical structures of maize plants. The method was tested using 40 samples from field observations and showed high accuracy in the segmentation of both organs (F-score =0.8207) and plants (F -score =0.9909). The effectiveness of terrestrial LiDAR for phenotyping at organ (including leaf area and stem position) and individual plant (including individual height and crown width) levels in field environ-ments was evaluated. The accuracies of derived stem position (position error =0.0141 m), plant height (R2 >0.99), crown width (R2 >0.90), and leaf area (R2 >0.85) allow investigating plant structural and func-tional phenotypes in a high-throughput way. This CNN-based solution overcomes the major challenges in organ-level phenotypic trait extraction associated with the organ segmentation, and potentially con-tributes to studies of plant phenomics and precision agriculture. (c) 2022 2022 Crop Science Society of China and Institute of Crop Science, CAAS. Production and hosting by Elsevier B.V. on behalf of KeAi Communications Co., Ltd. This is an open access article under the CC BY-NC -ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).