A Fast Video Object Segmentation Method Based on Inductive Learning and Transductive Reasoning

With the development of the multimedia and Internet technology, people can easily shoot a large number of video and photos through camera equipment and upload to the Internet now. The computer vision is a very important research field, whose goal is to make the computers to understand the content of video like humans. Video Object Segmentation(VOS) is one of the basic tasks in computer vision. The video object segmentation task aims to automatically obtain the pixel-level mask corresponding to the object of interest in the video sequence by the algorithm. Video object segmentation has various important applications, including automatic driving, video surveillance, video editing, video understanding and so on. Video object segmentation is a highly challenging problem since appearance change, scale change, target occlusion, non-rigid shape change, background interference etc. The existing methods can be divided into two categories according to the different utilization of the real label of the first frame of a given video. One is based on online inductive learning, the other is based on transductive reasoning learning. Previous methods based on online inductive learning finetune the whole network on the given initial frame segmentation mask in the inference stage in order to obtain accurate results, resulting in large time consumption and difficult to meet the real-time requirements. In addition, previous methods based on transductive reasoning need to use a large amount of synthetic data or annotation data when modeling video temporal reasoning rules, which increases the cost of algorithm training. In order to make full use of the advantages of two kinds of algorithms based on online inductive learning and transductive reasoning, and avoid the disadvantages of the two methods, a fast video object segmentation method combining online inductive learning and transductive reasoning is proposed in this paper. Specifically, the transductive reasoning branch is pre-trained by the video prediction self-supervised learning method. It can model the short-term temporal transformation and motion information of the video through the images and segmentation results of the few previous frames, so as to infer the segmentation results of the current frame. The learned temporal features can guide the network to improve the stability of video segmentation. In the pre-training process of the transductive reasoning branch, only the original video data need to be used without any additional synthetic data or manual annotation. The online inductive learning branch is trained online according to the reference frame to learn the appearance discriminant feature of the target and provide long-term appearance discriminant power. In order to improve the test speed, different from the previous methods, the inductive learning branch proposed in this paper does not use the first frame to fine tune the whole network online, but updates a very lightweight template network online to make the template network produce a rough object segmentation mask. This rough segmentation mask is used as the attention map of attention mechanism, and the final segmentation result is generated through the decoding module together with the temporal features and the image features of the current frame. Several experiments are carried out on three challenging datasets (i.e., DAVIS-2016, DAVIS-2017 and YouTube-VOS) to show that our method achieves competitive performance against the state-of-the-arts. © 2022, Science Press. All right reserved.