Combining CNN and transformers for full-reference and no-reference image quality assessment

被引:10
|
作者
Zeng, Chao [1 ]
Kwong, Sam [2 ]
机构
[1] Hubei Univ, Sch Artificial Intelligence, Wuhan, Peoples R China
[2] City Univ Hong Kong, Dept Comp Sci, Hong Kong, Peoples R China
关键词
Image quality assessment; Convolutional neural network; Transformers; Non-local information; STATISTICS;
D O I
10.1016/j.neucom.2023.126437
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Most deep learning approaches for image quality assessment use regression from deep features extracted by CNN (Convolutional Neural Networks). However, non-local information is usually neglected in exist-ing methods. Motivated by the recent success of transformers in modeling contextual information, we propose a hybrid framework that utilizes a vision transformer backbone to extract features and a CNN decoder for quality estimation. We propose a shared feature extraction scheme for both FR and NR set-tings. A two-branch structured attentive quality predictor is devised for quality prediction. Evaluation experiments on various IQA datasets, including LIVE, CSIQ and TID2013, LIVE-Challenge, KADID-10 K, and KONIQ-10 K, show that our proposed models achieve outstanding performance for both FR and NR settings.& COPY; 2023 Published by Elsevier B.V.
引用
收藏
页数:11
相关论文
共 50 条
  • [31] Full-reference image quality assessment based on image segmentation with edge feature
    Shi, Zaifeng
    Zhang, Jiaping
    Cao, Qingjie
    Pang, Ke
    Luo, Tao
    SIGNAL PROCESSING, 2018, 145 : 99 - 105
  • [32] Neural Network-Based Full-Reference Image Quality Assessment
    Bosse, Sebastian
    Maniry, Dominique
    Mueller, Klaus-Robert
    Wiegand, Thomas
    Samek, Wojciech
    2016 PICTURE CODING SYMPOSIUM (PCS), 2016,
  • [33] Full-Reference Predictive Modeling of Subjective Image Quality Assessment with ANFIS
    El-Alfy, El-Sayed M.
    Riaz, Mohammed Rehan
    AGENTS AND ARTIFICIAL INTELLIGENCE, ICAART 2014, 2015, 8946 : 296 - 311
  • [34] RVSIM: a feature similarity method for full-reference image quality assessment
    Yang, Guangyi
    Li, Deshi
    Lu, Fan
    Liao, Yue
    Yang, Wen
    EURASIP JOURNAL ON IMAGE AND VIDEO PROCESSING, 2018,
  • [35] Full-Reference Image Quality Assessment Measure Based on Color Distortion
    Seghir, Zianou Ahmed
    Hachouf, Fella
    COMPUTER SCIENCE AND ITS APPLICATIONS, CIIA 2015, 2015, 456 : 66 - 77
  • [36] Hybrid Feature Similarity Approach to Full-Reference Image Quality Assessment
    Okarma, Krzysztof
    COMPUTER VISION AND GRAPHICS, 2012, 7594 : 212 - 219
  • [37] RVSIM: a feature similarity method for full-reference image quality assessment
    Guangyi Yang
    Deshi Li
    Fan Lu
    Yue Liao
    Wen Yang
    EURASIP Journal on Image and Video Processing, 2018
  • [38] Dynamic Receptive Field Generation for Full-Reference Image Quality Assessment
    Kim, Woojae
    Nguyen, Anh-Duc
    Lee, Sanghoon
    Bovik, Alan Conrad
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2020, 29 : 4219 - 4231
  • [39] Machine learning to design full-reference image quality assessment algorithm
    Charrier, Christophe
    Lezoray, Olivier
    Lebrun, Gilles
    SIGNAL PROCESSING-IMAGE COMMUNICATION, 2012, 27 (03) : 209 - 219
  • [40] Full-reference image quality assessment using statistical local correlation
    Ding, Yong
    Wang, Shaoze
    Zhang, Dong
    ELECTRONICS LETTERS, 2014, 50 (02) : 79 - 80