DACNN: Blind Image Quality Assessment via a Distortion-Aware Convolutional Neural Network

被引:50
作者
Pan, Zhaoqing [1 ]
Zhang, Hao [1 ]
Lei, Jianjun [1 ]
Fang, Yuming [2 ]
Shao, Xiao [3 ]
Ling, Nam [4 ]
Kwong, Sam [5 ]
机构
[1] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[2] Jiangxi Univ Finance & Econ, Sch Informat Technol, Nanchang 330032, Jiangxi, Peoples R China
[3] Nanjing Univ Informat Sci & Technol, Sch Comp & Software, Nanjing 210044, Peoples R China
[4] Santa Clara Univ, Dept Comp Sci & Engn, Santa Clara, CA 95053 USA
[5] City Univ Hong Kong, Dept Comp Sci, Hong Kong, Peoples R China
关键词
Distortion; Feature extraction; Image quality; Databases; Fuses; Semantics; Knowledge engineering; Blind image quality assessment; distortion-aware network; Siamese network; fusion network; DATABASE;
D O I
10.1109/TCSVT.2022.3188991
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Deep neural networks have achieved great performance on blind Image Quality Assessment (IQA), but it is still challenging for using one network to accurately predict the quality of images with different distortions. In this paper, a Distortion-Aware Convolutional Neural Network (DACNN) is proposed for blind IQA, which works effectively for not only synthetically distorted images but also authentically distorted images. The proposed DACNN consists of a distortion aware module, a distortion fusion module, and a quality prediction module. In the distortion aware module, a Siamese network-based pretraining strategy is proposed to design a synthetic distortion-aware network for full learning the synthetic distortions, and an authentic distortion-aware network is used for extracting the authentic distortions. To efficiently fuse the learned distortion features, and make the network pay more attention to the essential features, a weight-adaptive fusion network is proposed to adaptively adjust the weight of each distortion. Finally, the quality prediction module is adopted to map the fused features to a quality score. Extensive experiments on four authentic IQA databases and four synthetic IQA databases have proved the effectiveness of the proposed DACNN.
引用
收藏
页码:7518 / 7531
页数:14
相关论文
共 52 条
[1]   (ASNA) An Attention-based Siamese-Difference Neural Network with Surrogate Ranking Loss function for Perceptual Image Quality Assessment [J].
Ayyoubzadeh, Seyed Mehdi ;
Royat, Ali .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2021, 2021, :388-397
[2]   On the use of deep learning for blind image quality assessment [J].
Bianco, Simone ;
Celona, Luigi ;
Napoletano, Paolo ;
Schettini, Raimondo .
SIGNAL IMAGE AND VIDEO PROCESSING, 2018, 12 (02) :355-362
[3]   Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment [J].
Bosse, Sebastian ;
Maniry, Dominique ;
Mueller, Klaus-Robert ;
Wiegand, Thomas ;
Samek, Wojciech .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2018, 27 (01) :206-219
[4]   Learning a similarity metric discriminatively, with application to face verification [J].
Chopra, S ;
Hadsell, R ;
LeCun, Y .
2005 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOL 1, PROCEEDINGS, 2005, :539-546
[5]  
Deng J, 2009, PROC CVPR IEEE, P248, DOI 10.1109/CVPRW.2009.5206848
[6]   Perceptual Quality Assessment for Screen Content Images by Spatial Continuity [J].
Fang, Yuming ;
Du, Rengang ;
Zuo, Yifan ;
Wen, Wenying ;
Li, Leida .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2020, 30 (11) :4050-4063
[7]   Perceptual Quality Assessment of Smartphone Photography [J].
Fang, Yuming ;
Zhu, Hanwei ;
Zeng, Yan ;
Ma, Kede ;
Wang, Zhou .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :3674-3683
[8]   No-Reference Quality Assessment of Contrast-Distorted Images Based on Natural Scene Statistics [J].
Fang, Yuming ;
Ma, Kede ;
Wang, Zhou ;
Lin, Weisi ;
Fang, Zhijun ;
Zhai, Guangtao .
IEEE SIGNAL PROCESSING LETTERS, 2015, 22 (07) :838-842
[9]   Massive Online Crowdsourced Study of Subjective and Objective Picture Quality [J].
Ghadiyaram, Deepti ;
Bovik, Alan C. .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2016, 25 (01) :372-387
[10]   Visual Importance and Distortion Guided Deep Image Quality Assessment Framework [J].
Guan, Jingwei ;
Yi, Shuai ;
Zeng, Xingyu ;
Cham, Wai-Kuen ;
Wang, Xiaogang .
IEEE TRANSACTIONS ON MULTIMEDIA, 2017, 19 (11) :2505-2520