MC360IQA: A Multi-channel CNN for Blind 360-Degree Image Quality Assessment

被引:146
作者
Sun, Wei [1 ,2 ]
Min, Xiongkuo [1 ,2 ]
Zhai, Guangtao [1 ,2 ]
Gu, Ke [3 ]
Duan, Huiyu [1 ,2 ]
Ma, Siwei [4 ]
机构
[1] Shanghai Jiao Tong Univ, AI Inst, MoE Key Lab Artificial Intelligence, Shanghai 200240, Peoples R China
[2] Shanghai Jiao Tong Univ, Inst Image Commun & Informat Proc, Shanghai 200240, Peoples R China
[3] Beijing Univ Technol, Fac Informat Technol, Beijing Key Lab Computat Intelligence & Intellige, Beijing 100124, Peoples R China
[4] Peking Univ, Inst Digital Media, Sch Elect Engn & Comp Sci, Beijing 100871, Peoples R China
基金
中国博士后科学基金; 中国国家自然科学基金;
关键词
Image quality; Image coding; Distortion; Streaming media; Quality assessment; Indexes; 360-degree images; image quality assessment; convolution neural network; hyper-structure; FREE-ENERGY PRINCIPLE; STRUCTURAL SIMILARITY; PREDICTION; INDEX;
D O I
10.1109/JSTSP.2019.2955024
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
360-degree images/videos have been dramatically increasing in recent years. The characteristic of omnidirectional-view results in high resolution of 360-degree images/videos, which makes them difficult to be transported and stored. To deal with the problem, video coding technologies are used to compress the omnidirectional content but they will introduce the compression distortion. Therefore, it is important to study how popular coding technologies affect the quality of 360-degree images. In this paper, we present a study on both subjective and objective quality assessment of compressed virtual reality (VR) images. We first build a compressed VR image quality (CVIQ) database including 16 reference images and 528 compressed ones with three prevailing coding technologies. Then, we propose a multi-channel convolution neural network (CNN) for blind 360-degree image quality assessment (MC360IQA). To be consistent with the visual content seen in the VR device, we project each 360-degree image into six viewport images, which are adopted as inputs of the proposed model. MC360IQA consists of two parts, a multi-channel CNN and an image quality regressor. The multi-channel CNN includes six parallel hyper-ResNet34 networks, where the hyper structure is used to incorporate the features from intermediate layers. The image quality regressor fuses the features and regresses them to final scores. The experimental results show that our model achieves the best performance among the state-of-art full-reference (FR) and no-reference (NR) image quality assessment (IQA) models on the CVIQ database and other available 360-degree IQA database.
引用
收藏
页码:64 / 77
页数:14
相关论文
共 75 条
[1]  
[Anonymous], 2013, PREPRINT ARXIV 1308
[2]  
[Anonymous], IEEE T CIRCUITS SYST
[3]  
[Anonymous], METH SUBJ ASS QUAL T
[4]  
[Anonymous], WS PSNR 360 VIDEO OB
[5]  
[Anonymous], P IEEE INT C MULT EX
[6]  
[Anonymous], VR DAT REP
[7]  
[Anonymous], IEEE T CIRCUITS SYST
[8]  
[Anonymous], 2018, P IEEE INT C MULT EX
[9]  
Boski M, 2017, 2017 10TH INTERNATIONAL WORKSHOP ON MULTIDIMENSIONAL (ND) SYSTEMS (NDS)
[10]   Accuracy and cross-calibration of video quality metrics: new methods from ATIS/T1A1 [J].
Brill, MH ;
Lubin, J ;
Costa, P ;
Wolf, S ;
Pearson, J .
SIGNAL PROCESSING-IMAGE COMMUNICATION, 2004, 19 (02) :101-107