Multi-scale Spatial-Temporal Feature Aggregating for Video Salient Object Segmentation

This paper proposes an algorithm based on supervised deep convolutional neural networks (CNNs), which fully extracts and fuses spatial-temporal information of frames to enhance the video saliency detection performance. Conventional video saliency detection methods that exist some problems (e.g. cumulative miscellaneous information, acquisition of spatial-temporal information separately, time-consuming algorithm, etc.), not only cannot make the spatial temporal information fully integrated, but cannot meet the real-time requirements. A Multi-scale Spatial Feature Extraction Module (MSFEM) based on deep learning is first designed for simultaneously extracting spatial features at multiple scales. Then, we further fuse the spatial-temporal features of the frames by adopting Multi-scale Spatial-Temporal Feature Refine Module (MSTFRM) to take full advantage of spatial-temporal information in order to achieve high-quality detections with strong spatial-temporal saliency consistency. After that, the network adopts end-to-end network training and testing to avoid unnecessary time overhead caused by pretreatment. To validate the method, we make comprehensive, quantitative evaluations between our method and 8 state-of-the-art techniques. All the results demonstrate our method's advantages in terms of accuracy and reliability.