Coarse-to-Fine Feedback Guidance Based Stereo Image Quality Assessment Considering Dominant Eye Fusion

被引:2
作者
Chang, Yongli [1 ]
Li, Sumei [1 ]
Liu, Anqi [1 ]
Jin, Jie [1 ]
Xiang, Wei [2 ]
机构
[1] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[2] La Trobe Univ, Sch Engn & Math Sci, Melbourne, Vic 3086, Australia
基金
中国国家自然科学基金;
关键词
Visualization; Feature extraction; Image quality; Adaptive systems; Streaming media; Data mining; Bidirectional control; Stereo image quality assessment; coarse-to-fine feedback guidance; bi-directional parallax attention mechanism; convolutional neural network; FUNCTIONAL ARCHITECTURE; PREDICTION;
D O I
10.1109/TMM.2023.3242550
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Considering that the human brain always follows a coarse-to-fine (low-to-high spatial frequency) visual processing and fusion mechanism, we propose a coarse-to-fine feedback guidance based stereo image quality assessment (SIQA) network which considers a coarse-to-fine feedback guidance and adaptive dominant eye mechanism. The proposed network consists of two main sub-network streams, each of which has three branches to extract low, middle and high spatial frequency information in parallel. To better realize the guidance of the high-level features in the low spatial frequency branch to the low-level features in the high spatial frequency branch, an information feedback guidance module (IFGM) is proposed, which realizes a top-down guidance mechanism in each sub-network stream. Simultaneously, according to the theory of ocular dominance in human visual system (HVS), we design an adaptive bi-directional parallax-based binocular fusion module (BPBFM), which synthesizes two types of fusion feature by taking the left and right view features as dominant eye input. Furthermore, in order to obtain the better perceptual quality of stereo images, we design a weighted fusion strategy to weigh the quality scores from the two types of fusion features obtained by using an ensemble model with two multi-layer perceptrons (MLPs). The experimental results on four public stereo image datasets show that the proposed method is superior to the mainstream metrics and achieves an excellent performance.
引用
收藏
页码:8855 / 8867
页数:13
相关论文
共 59 条
  • [1] Blind Quality Assessment of Screen Content Images Via Macro-Micro Modeling of Tensor Domain Dictionary
    Bai, Yongqiang
    Zhu, Zhongjie
    Jiang, Gangyi
    Sun, Huifang
    [J]. IEEE TRANSACTIONS ON MULTIMEDIA, 2021, 23 : 4259 - 4271
  • [2] Perceptual Quality Assessment of Cartoon Images
    Chen, Hangwei
    Chai, Xiongli
    Shao, Feng
    Wang, Xuejin
    Jiang, Qiuping
    Meng, Xiangchao
    Ho, Yo-Sung
    [J]. IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 140 - 153
  • [3] No-reference perceptual quality assessment of stereoscopic images based on binocular visual characteristics
    Chen, Lei
    Zhao, Jiying
    [J]. SIGNAL PROCESSING-IMAGE COMMUNICATION, 2019, 76 : 1 - 10
  • [4] No-Reference Quality Assessment of Natural Stereopairs
    Chen, Ming-Jun
    Cormack, Lawrence K.
    Bovik, Alan C.
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2013, 22 (09) : 3379 - 3391
  • [5] Semi-Reference Sonar Image Quality Assessment Based on Task and Visual Perception
    Chen, Weiling
    Gu, Ke
    Zhao, Tiesong
    Jiang, Gangyi
    Le Callet, Patrick
    [J]. IEEE TRANSACTIONS ON MULTIMEDIA, 2021, 23 : 1008 - 1020
  • [6] Ding Y., 2019, P IEEE VIS COMM IM P, P1
  • [7] Stereoscopic image quality assessment by analysing visual hierarchical structures and binocular effects
    Ding, Yong
    Zhao, Yang
    Chen, Xiaodong
    Zhu, Xiaolei
    Andrey, Krylov
    [J]. IET IMAGE PROCESSING, 2019, 13 (10) : 1608 - 1615
  • [8] No-Reference Stereoscopic Image Quality Assessment Using Convolutional Neural Network for Adaptive Feature Extraction
    Ding, Yong
    Deng, Ruizhe
    Xie, Xin
    Xu, Xiaogang
    Zhao, Yang
    Chen, Xiaodong
    Krylov, Andrey S.
    [J]. IEEE ACCESS, 2018, 6 : 37595 - 37603
  • [9] Learning a No-Reference Quality Predictor of Stereoscopic Images by Visual Binocular Properties
    Fang, Yuming
    Yan, Jiebin
    Wang, Jiheng
    Liu, Xuelin
    Zhai, Guangtao
    Le Callet, Patrick
    [J]. IEEE ACCESS, 2019, 7 : 132649 - 132661
  • [10] Stereoscopic image quality assessment by deep convolutional neural network
    Fang, Yuming
    Yan, Jiebin
    Liu, Xuelin
    Wang, Jiheng
    [J]. JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 2019, 58 : 400 - 406