Incorporating illumination constraints in deformable models for shape from shading and light direction estimation

In this paper, we present a method for the integration of nonlinear holonomic constraints in deformable models and its application to the problems of shape and illuminant direction estimation from shading. Experimental results demonstrate that our method performs better than previous Shape from Shading algorithms applied to images of Lambertian objects under known illumination. It is also more general as it can be applied to non-Lambertian surfaces and it does not require knowledge of the illuminant direction. In this paper, 1) we first develop a theory for the numerically robust integration of nonlinear holonomic constraints within a deformable model framework. In this formulation, we use Lagrange multipliers and a Baumgarte stabilization approach [2]. 2) We also describe a fast new method for the Computation of constraint based forces, in the case of high numbers of local parameters. 3) We demonstrate how any type of illumination constraint, from the simple Lambertian model to more complex highly nonlinear models [27], [29] can be incorporated in a deformable model framework. 4) We extend our method to work when the direction of the light source is not known. We couple our shape estimation method with a method for light estimation, in an iterative process, where improved shape estimation results in improved light estimation and vice versa. 5) We perform a series of experiments on both synthetic and real data. The synthetic data come from a standardized set of images [43]. Our results compare favorably with results of previous SfS algorithms on the same data and our light direction estimation to a previous method by Zheng and Chellapa [44].