Dynamic visual servo control methods for continuous operation of a fruit harvesting robot working throughout an orchard

Fruit-picking robots are crucial for achieving efficient orchard harvesting. To genuinely meet the commercial production needs of farmers, the new generation of fruit-picking robots must be capable of demonstrating complete and continuous observation, movement, and picking behaviors throughout complex orchards, akin to real human employees. This poses systematic challenges, as many prior researches have focused solely on a part of the continuous operation of the entire orchard, such as fruit positioning, navigation, path planning, or grasping. These isolated basic functions are important but insufficient for fulfilling operational requirements on a macro scale and continuous situation. Developing an efficient control method for each basic module and constructing their internal coordination is vital for transitioning a harvesting robot from a functional prototype to a practical machine. In this context, this study tackles the visual servo control problem for efficient locomotion, picking, and their seamless integration. A set of vision algorithms for locomotion destination estimation, realtime self-positioning, and dynamic harvesting is proposed. Additionally, a solid coordination mechanism for continuous locomotion and picking behavior is established. Each method offers distinct advantages, such as improved accuracy, adaptability to varying conditions, and enhanced picking efficiency, enabling the robot to operate autonomously and continuously. Comprehensive field experiments validated the soundness of the methods. The primary contribution of this study lies in addressing the challenge of continuous operation in an entire orchard as a systematic problem and providing new insights into control methods for the future development of highly autonomous, practical, and user-oriented fruit harvesting systems.