Blind quality assessment of omnidirectional videos using spatio-temporal convolutional neural networks

Panoramic videos are different from 2D videos in the spherical viewing mode. Normally, 2D videos are viewed by consumers on flat screen televisions, while panoramic videos need specific head-mounted displays to create virtual reality experience. Therefore, it is meaningful to study a quality evaluation model suitable for panoramic videos, which will bring a better quality of experience in the field of visual consumption. Unfortunately, to the best of our knowledge, there are few omnidirectional video quality assessments especially in no-reference quality evaluation modes. Besides, the existing 2D quality evaluation methods cannot well evaluate panoramic content quality. Based on above motivation and the vacancy of related research, we propose a blind deep learning-driven quality evaluation framework for panoramic videos to address the correlative issues. From the perspective of the viewport format, we eliminate the sampling distortion of panoramic videos in equirectangular projection (ERP) format via cubemap projection (CMP) format projection. In addition, we use a two-stream convolutional network to fully extract the intra-frame and inter-frame (defined as spatio-temporal) information for more comprehensive modeling of panoramic video features. Experimental results show that our method is superior to the existing full-reference and no-reference methods in solving the task of panoramic video quality assessment.