Image moments-based visual servoing control of bagged agricultural materials handling robot

Manual handling is less efficient and sometimes even hazardous to humans in many areas, for example, agriculture. Using robots in those areas not only avoids human contact with such dangerous agricultural materials but also improves working efficiency. The motion of a robot is controlled using a technique called visual servoing that uses feedback information extracted from a vision sensor. In this study, a visual servoing method was proposed based on learning features and image moments for 3D targets to solve the problem of image moments-based visual servoing. A Gaussian process regression model was used to map the relationship between the image moment invariants and the rotational angles around the X- and Y-axes of the camera frame (denoted as gamma and beta). To obtain maximal decoupled structure and minimal nonlinearities of the image Jacobian matrix, it was assumed two image moment features, which are linearly proportional to gamma and beta. Combining the four image moment features of the normalized centroid coordinates, area, and orientation angle, a 6-DOF image moment-based visual servoing controller for the agricultural material handling robot was designed. Using this method, the problem of visual servoing task failure due to the singularity of the Jacobian matrix was solved, and it also had a better convergence effect for the part of the target image beyond the field of view from the initial pose and large displacement visual servoing system. The proposed algorithm was validated by carrying out experiments tracking bagged flour in a six-degree-of-freedom robotic system. The final displacement positioning accuracy reached the millimeter level and the direction angle positioning accuracy reached the level of 0.1 degrees. The method still has a certain convergence effect when the target image is beyond the field of view from the initial pose. The experimental results have been presented to show the adequate behavior of the presented approach in robot handling operations. It provides reference for the application of visual servoing technology in the field of agricultural robots and has important theoretical significance and practical value.