Graph cut based segmentation of soft shadows for seamless removal and augmentation

This paper introduces a new concept within shadow segmentation for usage in shadow removal and augmentation through construction of a multiplicity alpha overlay shadow model. Previously, an image was considered to consist of shadow and non-shadow regions. This makes it difficult to seamlessly remove shadows and insert augmented shadows that overlap real shadows. We construct a model that accounts for sunlit, umbra and penumbra regions by estimating the degree of shadow. The model is based on theories about color constancy, daylight, and the geometry that causes penumbra. A graph cut energy minimization is applied to estimate the alpha parameter. Overlapping shadow augmentation and removal is also demonstrated. The approach is demonstrated on natural complex image situations. The results are convincing, and the quality of augmented shadows overlapping real shadows and removed shadows depends on the quality of the estimated alpha gradient in penumbra.