Spherical Convolution-based Saliency Detection for FoV Prediction in 360-degree Video Streaming

Field of view (FoV) prediction is a crucial issue in 360 degrees video streaming, which is the basis for selectively transmitting panoramic videos to reduce bandwidth. The saliency feature is a very important part of FoV prediction. The saliency area identifies a user's region of interest (RoI) and reflects the user's viewing behavior preference. The regular convolutional neural network (CNN) cannot effectively extract the spatial representation of panoramic video content because significant geometric distortion will be introduced after panoramic video projection, especially in polar regions. In this paper, we propose a depth neural network model based on spherical convolution, which can learn the spatial features of the 360 degrees videos by encoding the distortion invariance into the architecture of CNNs. A series of experiments on the public 360 degrees video saliency dataset show the proposed model outperforms the existing saliency models. Finally, we embed the proposed saliency network into a popular FoV prediction framework and propose a complete FoV prediction framework for 360 degrees video streaming.