A Visual-Inertial Servoing Method for Tracking Object with Two Landmarks and an Inertial Measurement Unit

This paper introduces a robot visual-inertial tracking algorithm for a robot manipulator intended to track an object using inertial sensors incorporated into the object. To create this algorithm, the inertial Jacobian is first newly defined in order to show the relationship between an angle set velocity vector of the object and the angular velocity vector of the robot tip. Then, the inertial Jacobian is combined with the conventional image Jacobian. Therefore, the proposed algorithm requires only two landmarks with the help of an inertial measurement unit to track a moving object with six degrees of freedom, while at least three landmarks are required in the conventional stereo visual servoing algorithm. Further, the possession of a multi-rate controller allows the integration system to out-perform conventional systems in the tracking of an object's attitude change. A suggested application of the proposed method is tracking and selection of a container from a shipping vessel that is being affected by large waves. Simulations and experiments were conducted to verify the feasibility of the proposed methodology.