ADAPTIVE ROBOTIC VISUAL TRACKING - THEORY AND EXPERIMENTS

This paper addresses the use of a vision sensor in the feedback loop within the Controlled Active Vision framework. Using this framework, algorithms are proposed for the solution of the robotic (eye-in-hand configuration) visual tracking and servoing problem. We state the problem of visual tracking as a problem of combining control with computer vision. We use the sum-of-squared differences (SSD) optical flow for the computation of the vector of discrete displacements. The measurements can be derived either from a single big window or from multiple small windows. These displacements are fed to an adaptive controller (self-tuning regulator) that creates commands for a robot control system. The whole algorithm is based on the on-line estimation of the relative distance of the target with respect to the camera. An important contribution of this work is that it requires only partial knowledge of the relative distance of the target with respect to the camera. This fact obviates the need for off-line calibration of the eye-in-hand robotic system. We have implemented, both in simulation and in experiments, three different adaptive control schemes, and the results are presented in this paper. The computational complexity and the experimental results demonstrate that the proposed algorithms can be implemented in real time.