Robust Temporally-Coherent Strategy for Few-shot Video Instance Segmentation

Traditional video instance segmentation (VIS) aims to detect, segment, and track object instances from a known class set in videos. In real-world applications, however, video instance segmentation typically need to cope with novel-class instances and to fast adapt with a few labeled videos. In this work, we aim to tackle the task of few-shot video instance segmentation (FVIS), which is challenging due to large variations in object appearance and motion. We propose a robust temporally coherent strategy, termed as VTFA, based on a two-stage fine-tuning approach. VTFA enforces the instance segmentation of novel classes to be temporally smooth and reduces the classification bias between novel and base classes. The proposed Memory-aware Temporal Context Encoding Module (MTCE) in VTFA encodes the temporal context information, which contributes to the consistency in the final predictions. We also propose a loss named Instance-level Pair-wise Contrastive (IPC) Loss on both the novel and base classes to enhance the robustness of instance classification. To validate our method, we develop a YouTube-VIS-FS benchmark to compare our method with several baselines. The experimental evaluation shows that our strategy is superior or competitive to those strong baselines.