An illumination invariant framework for real-time foreground detection

The detection of interesting objects in a video sequence is a challenging problem, with many applications in automated surveillance and video analytic systems. Background subtraction methods have proven to be useful as a way to differentiate foreground objects from the background, but suffer from various shortcomings that hamper performance in practice. We propose a generalized framework to handle some of the greater challenges facing background modeling systems. This includes object persistence and reintegration, illumination robustness and resistance to environmental effects. In this paper we describe an interaction framework between an illumination invariant and a color-based model, with a novel feature clustering feedback that dynamically influences model updating and by proxy provides object validation. The framework was designed with a strong real-time emphasis, and the principles remain applicable by collapsing each module to a simpler real-time variant.