Fully automated proximal hyperspectral imaging system for high-resolution and high-quality in vivo soybean phenotyping

Hyperspectral imaging (HSI) is a prevalent method in crop phenotyping. Nevertheless, current HSI remote sensing techniques are compromised by changing ambient lighting conditions, long imaging distances, and comparatively low resolutions. Proximal HSI sensors such as LeafSpec were developed to improve the imaging quality. However, the application of proximal sensors remains contrained by their low throughput and intensive labor costs. Moreover, few automation solutions were available to use LeafSpec in phenotyping dicot plants. In this paper, a novel robotic system is presented as a sensor platform to operate LeafSpec to collect leaf-level hyperspectral images for in vivo phenotyping of soybean. A machine vision algorithm was developed to detect the top mature trifoliate and estimate the poses of the leaflets. A control and motion planning algorithm was developed for an articulated robotic manipulator to grasp the target leaflets. An experiment was conducted in March 2021 in a greenhouse with 64 soybean plants of 2 genotypes and 2 nitrogen treatments. The machine vision detected the target leaflets with a first trial success rate of 84.13% and an overall success rate of 90.66%. The robotic manipulator operated LeafSpec to image the target leaflets with a first trial success rate of 87.30% and an overall success rate of 93.65%. The average cycle time for one soybean plant was 63.20 s. The PLS predictions from the robot-collected data had an R-2 of 0.84 with the measured nitrogen content and an R-2 of 0.82 with the predictions from human-collected data. The results demonstrated the potential of applying the system for automated in vivo leaf-level HSI for soybean phenotyping in the field.