No-Reference Image Quality Assessment via Multibranch Convolutional Neural Networks

被引：24
作者：
Pan Z. [1 ]
Yuan F. [1 ]
Wang X. [2 ]
Xu L. [3 ]
Shao X. [4 ]
Kwong S. [5 ]
机构：
[1] the School of Electrical and Information Engineering, Tianjin University, Tianjin
[2] the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen
[3] the Key Laboratory of Solar Activity, National Astronomical Observatories,, Chinese Academy of Sciences,, Beijing
[4] the School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing
[5] the Department of Computer Science, City University of Hong Kong
来源：
IEEE Transactions on Artificial Intelligence | 2023年 / 4卷 / 01期
基金：
中国国家自然科学基金;
关键词：
Hierarchical feature merge module (HFMM); multibranch CNN; no-reference image quality assessment (NR-IQA); position features;
D O I：
10.1109/TAI.2022.3146804
中图分类号：
学科分类号：
摘要：
No-reference image quality assessment (NR-IQA) aims to evaluate image quality without using the original reference images. Since the early NR-IQA methods based on distortion types were only applicable to specific distortion scenarios, and lack of practicality, it is challenging to designing a universal NR-IQA method. In this article, a multibranch convolutional neural network (MB-CNN) based NR-IQA method is proposed, which includes a spatial-domain feature extractor, a gradient-domain feature extractor, and a weight mechanism. The spatial-domain feature extractor aims to extract the distortion features from the spatial domain. The gradient-domain feature extractor is used to guide the spatial-domain feature extractor to pay more attention to the distortions of the structure information. Particularly, the spatial-domain feature extractor uses the hierarchical feature merge module to realize multiscale feature representation, and the gradient-domain feature extractor uses pyramidal convolution to extract the multiscale structure information of the distorted image. In addition, a position vector is proposed to build the weight mechanism by considering the position relationships between patches and its entire image for improving the final prediction performance. We conduct the experiments on five representative databases: LIVE, TID2013, CSIQ, LIVE MD and Waterloo Exploration Database, and the experimental results show that the proposed NR-IQA method achieves the state-of-the-art performance, which demonstrate the effectiveness of our proposed NR-IQA method. The code ofthe proposed MB-CNN will be released at https://github.com/NUIST-Videocoding/MB-CNN. © 2020 IEEE.
引用
收藏
页码：148 / 160
页数：12
相关论文
共 50 条
  • [1] Piazzo L., Raguso M.C., Calzoletti L., Seu R., Altieri B., Least squares time-series synchronization in image acquisition systems, IEEE Trans. Image Process., 25, 9, pp. 4458-4468, (2016)
  • [2] Zhu S., Li M., Chen C., Liu S., Zeng B., Cross-space distortion directedcolorimagecompression, IEEETrans.Multimedia, 20, 3, pp. 525-538, (2018)
  • [3] Zhang M., Desrosiers C., High-quality image restoration using lowrankpatchregularizationandglobalstructuresparsity, IEEETrans.Image Process., 28, 2, pp. 868-879, (2019)
  • [4] Wang S., Luo G., Naturalness preserved image enhancement using a priorimulti-layerlightnessstatistics, IEEETrans.ImageProcess., 27, 2, pp. 938-948, (2018)
  • [5] Wang Z., Bovik A.C., Sheikh H.R., Simoncelli E.P., Image quality assessment: From error visibility to structural similarity, IEEE Trans. Image Process., 13, 4, pp. 600-612, (2004)
  • [6] Wang Z., Simoncelli E.P., Bovik A.C., Multiscale structural similarity for image quality assessment, Proc. 37th Asilomar Conf. Signals, Syst. Comput., 2, pp. 1398-1402, (2003)
  • [7] Sheikh H.R., Bovik A.C., Image information and visual quality, IEEE Trans. Image Process., 15, 2, pp. 430-444, (2006)
  • [8] Sheikh H.R., Bovik A.C., An information fidelity criterion for image quality assessment using natural scene statistics, IEEE Trans. Image Process., 14, 12, pp. 2117-2128, (2005)
  • [9] Zhang L., Zhang L., Mou X., Zhang D., FSIM: A feature similarity index for image quality assessment, IEEE Trans. Image Process., 20, 8, pp. 2378-2386, (2011)
  • [10] Wu J., Liu Y., Shi G., Lin W., Saliency change based reduced reference image quality assessment, Proc. IEEE Vis. Commun. Image Process., pp. 1-4, (2017)
12345