A scale selection principle for estimating image deformations

A basic functionality of a vision system concerns the ability to compute deformation fields between different images of the same physical structure. This article advocates the need for incorporating an explicit mechanism for automatic scale selection in this context, in algorithms for computing descriptors such as optic flow and for performing stereo matching. A basic reason why such a mechanism is essential is the fact that in a coarse-to-fine propagation of disparity or flow information, it is not necessarily the case that the most accurate estimates are obtained at the finest scales. The existence of interfering structures at fine scales may make it impossible to accurately match the image data at fine scales. A systematic methodology for approaching this problem is proposed, by estimating the uncertainty in the computed flow estimate at each scale, and then selecting deformation estimates from the scales that minimize a (suitably normalized) measure of uncertainty over scales. A specific implementation of this idea is presented for a region based differential flow estimation scheme, which besides a hierarchical and iterative coarse-to-fine computation of flow updates, involves explicit use of confidence values for how field averaging. It is shown that the integrated scale selection and flow estimation algorithm has the qualitative properties of leading to the selection of coarser scales for larger size image structures and increasing noise level, whereas it leads to the selection of finer scales in the neighbourhood of flow field discontinuities. The latter property may serve as an indicator when detecting flow field discontinuities and occlusions. (C) 1998 Elsevier Science B.V. All rights reserved.