AVScreen: a real-time video augmentation method

We present a tool for video augmentation in real-time, which we name the augmentation virtual screen (AVScreen). AVScreen is useful for developing advertisements, commercials, music videos, movies, etc. The main challenges for augmenting videos, in contrast to fixed images, is that moving objects in the foreground may occlude the region to be augmented in the background and that the composition can be affected by camera movements. Therefore, we use a procedure for foreground–background video segmentation in order to deal with such occlusions. Comparisons with foreground–background video segmentation methods of the state of the art in both accuracy and computational efficiency support our choice: we reduce around 70 % of the segmentation error in a popular benchmark database and achieve real-time performance. Moreover, a new stabilization method to augment unstable camera videos is presented. For augmenting video shots, we present an efficient graph-based method for panorama (mosaic) computation. The real-time performance is reached by implementing high computational demanding procedures in GPU. The frame rate of our method is 18 frames per second for a video size of 640 × 480 pixels.