Research on 3D reconstruction of fruit tree and fruit recognition and location method based on RGB-D camera

In order to provide a scientific and reliable technical guidance for fruit harvesting robot in orchard, a method was proposed in this paper to reconstruct 3D image for apple tree and carry out recognition and location for each apple fruit. Firstly, the color image and depth image of the fruit trees were taken by an RGB-D camera, and the 3D reconstruction of each fruit tree was carried out by fusing its color and depth information. Then, 3D point cloud of the fruit region were segmented from tree's point cloud by applying the color threshold of R-G. Finally, the 3D shape of each fruit point cloud was extracted and its 3D spatial position information and radius were also obtained by using iteratively the RANSAC (Random sample consensus) algorithm to fit each fruit to a pre-defined apple model. The experimental results showed that the proposed method of 3D reconstruction of apple tree and recognition and location of its fruits had good real-time performance and strong robustness. In the measurement range of 0.8~2.0 m, the correct recognition rates of fruits under frontlighting and backlighting conditions were 95.5% and 88.5% respectively, and the correct recognition rate was 87.4% in the case that the sheltered area of fruit point clouds was over 50%, besides, the average position calculation error of the fruit was 8.1 mm, and the average radius calculation error was 4.5 mm. © 2015, Chinese Society for Agricultural Machinery. All right reserved.