Fast Context Adaptation for Video Object Segmentation

In this paper we present an adaptation module for feature matching based Semi-automatic Video Object Segmentation methods (SVOS). Most current solutions to adapt SVOS methods during inference are slow or inefficient. Feature matching based methods use affinity between a set of reference and query features to segment a target in the current frame based on a reference. We propose an adaptation module working solely with the user supplied mask in the first frame of a video. Our adaptation of the matching module provides more reliable information to the model for segmentation in all the video frames and does not significantly increase inference time. The evaluation on both OVIS and DAVIS 17 datasets shows a significant improvement on the segmentation (respectively +2.9% and +1% of the Jaccard index). This demonstrates that our adaptation of the feature space provides a better matching between query and reference features.