Simple Combination of Appearance and Depth for Foreground Segmentation

In foreground segmentation, the depth information is robust to problems of the appearance information such as illumination changes and color camouflage; however, the depth information is not always measured and suffers from depth camouflage. In order to compensate for the disadvantages of the two pieces of information, we define an energy function based on the two likelihoods of depth and appearance backgrounds and minimize the energy using graph cuts to obtain a foreground mask. The two likelihoods are obtained using background subtraction. We use the farthest depth as the depth background in the background subtraction according to the depth information. The appearance background is defined as the appearance with a large likelihood of the depth background to eliminate appearances of foreground objects. In the computation of the likelihood of the appearance background, we also use the likelihood of the depth background for reducing false positives owing to illumination changes. In our experiment, we confirm that our method is sufficiently accurate for indoor environments using the SBM-RGBD 2017 dataset.