Visual Perception Method for Cotton-picking Robots Based on Fusion of Multi-view 3D Point Clouds

Traditional cotton-picking robots face visual perception challenges due to their reliance on single viewpoint and two-dimensional imagery. To address this, a multi-view 3D point cloud registration method was introduced, enhancing these robots' real-time 3D visual perception. Four fixed-pose Realsense D435 depth cameras were utilized to capture point cloud data of the cotton from multiple viewpoints. To ensure the quality of fusion registration, each camera underwent rigorous imaging distortion calibration and depth error adjustment before operation. With the help of AprilTags algorithm, the relative pose between the RGB imaging modules of the cameras and their AprilTag labels was calibrated, which clarified the transformation relationship between the coordinate systems of the RGB and stereo imaging modules. As a result,the transformations of point cloud coordinates between cameras can be deduced, ensuring accurate fusion and alignment. The findings showed that this method had an average global alignment error of 0.93 cm and took 0.025 s on average, highlighting its accuracy and efficiency against the commonly used methods. To cater to the real-time demands of cotton-picking robots, processes for point cloud acquisition, background filtering, and fusion registration were also optimized. Impressively, the algorithm's speed tops at 29. 85 f/s, meeting the real-time demands of the robot's perception system. © 2024 Chinese Society of Agricultural Machinery. All rights reserved.