A general framework for three-dimensional surface reconstruction by self-consistent fusion of shading and shadow features and its application to industrial quality inspection tasks

In this paper a novel framework for surface quality inspection of industrial parts based on three-dimensional surface reconstruction by self-consistent fusion of shading and shadow features is presented. Relying on the analysis of at least two pixel-synchronous greyscale images of the scene acquired under very different illumination conditions, this framework combines a shadow analysis of the first image of the scene, allowing for a determination of large-scale altitude differences on the surface at high accuracy, with a variational shape from shading scheme applied to the second image (and eventually to further images), estimating the surface gradients and altitude profile. In a first step, the result of shadow analysis is used for selecting a solution of the variational shape from shading scheme which is consistent with the average altitude difference derived by shadow analysis. In a second step, the detailed shadow structure is taken into account. An error term that aims at adjusting the altitude differences extracted from the reconstructed surface profile to those derived from shadow analysis is incorporated into the error function to be minimized by the variational shape from shading scheme. The second reconstruction step is initialized with the result of the first step. In contrast to existing shape from shading or photometric stereo approaches, our algorithm shows the advantage that it neither requires a very accurate knowledge of the reflectance function of the surface to be reconstructed, nor does it critically depend on the initialization. The described framework is applied to the three-dimensional reconstruction of metal sheet and raw cast iron surfaces in the context of industrial quality inspection.