Detecting Occlusions as an Inverse Problem

Occlusions generally become apparent when integrated over time because violations of the brightness-constancy constraint of optical flow accumulate in occluded areas. Based on this observation, we propose a variational model for occlusion detection that is formulated as an inverse problem. Our forward model adapts the brightness constraint of optical flow to emphasize occlusions by exploiting their temporal behavior, while spatio-temporal regularizers on the occlusion set make our model robust to noise and modeling errors. In terms of minimization, we approximate the resulting variational problem by a sequence of convex optimizations and develop efficient algorithms to solve them. Our experiments show the benefits of the proposed formulation, both forward model and regularizers, in comparison to the state-of-the-art techniques that detect occlusion as the residual of optical-flow estimation.