RANSP: Ranking attention network for saliency prediction on omnidirectional images

被引:0
|
作者
Zhu, Dandan [1 ]
Chen, Yongqing [2 ]
Min, Xiongkuo [3 ]
Zhu, Yucheng [3 ]
Zhang, Guokai [4 ]
Zhou, Qiangqiang [5 ]
Zhai, Guangtao [3 ]
Yang, Xiaokang [1 ,3 ]
机构
[1] Shanghai Jiao Tong Univ, Key Lab Artificial Intelligence, Minist Educ, Shanghai 200240, Peoples R China
[2] Hainan Air Traff Management Sub Bureau, Haikou 570000, Hainan, Peoples R China
[3] Shanghai Jiao Tong Univ, Inst Image Commun & Network Engn, Shanghai 200240, Peoples R China
[4] Univ Shanghai Sci & Technol, Sch Opt Elect & Comp Engn, Shanghai 200093, Peoples R China
[5] Shanghai Business Sch, Sch Informat & Comp, Shanghai 201400, Peoples R China
基金
中国国家自然科学基金;
关键词
Omnidirectional images; Saliency prediction; Ranking attention; Part-guided attention; Channel-wise feature maps; VISUAL-ATTENTION; COLOR; MODEL;
D O I
10.1016/j.neucom.2021.06.029
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Various convolutional neural network (CNN)-based methods have shown the ability to boost the performance of saliency prediction on omnidirectional images (ODIs). However, these methods are limited by sub-optimal accuracy, because not all the features extracted by the CNN model are useful for the final fine-grained saliency prediction. Some features are redundant and may have negative impact on the final fine-grained saliency prediction. To tackle this problem, we propose a novel Ranking Attention Network for saliency prediction (RANSP) of head fixations on ODIs. Specifically, the part-guided attention (PA) module and channel-wise feature (CF) extraction module are integrated in a unified framework and are trained in an end-to-end manner for fine-grained saliency prediction. To better utilize the channel-wise feature maps, we further propose a new Ranking Attention Module (RAM), which automatically ranks and selects these feature maps based on scores for fine-grained saliency prediction. Extensive experiments and ablation studies are conducted to show the effectiveness of our method for saliency prediction on ODIs. (c) 2021 Published by Elsevier B.V.
引用
收藏
页码:118 / 128
页数:11
相关论文
共 50 条
  • [31] Video attention prediction using gaze saliency
    Yanxiang Chen
    Gang Tao
    Qiangqiang Xie
    Minglong Song
    Multimedia Tools and Applications, 2019, 78 : 26867 - 26884
  • [32] Video attention prediction using gaze saliency
    Chen, Yanxiang
    Tao, Gang
    Xie, Qiangqiang
    Song, Minglong
    MULTIMEDIA TOOLS AND APPLICATIONS, 2019, 78 (19) : 26867 - 26884
  • [33] Decoupled dynamic group equivariant filter for saliency prediction on omnidirectional image
    Zhu, Dandan
    Zhang, Kaiwei
    Zhang, Guokai
    Zhou, Qiangqiang
    Min, Xiongkuo
    Zhai, Guangtao
    Yang, Xiaokang
    NEUROCOMPUTING, 2023, 518 : 111 - 121
  • [34] Delivery of omnidirectional video using saliency prediction and optimal bitrate allocation
    Cagri Ozcinar
    Nevrez İmamoğlu
    Weimin Wang
    Aljosa Smolic
    Signal, Image and Video Processing, 2021, 15 : 493 - 500
  • [35] Delivery of omnidirectional video using saliency prediction and optimal bitrate allocation
    Ozcinar, Cagri
    Imamoglu, Nevrez
    Wang, Weimin
    Smolic, Aljosa
    SIGNAL IMAGE AND VIDEO PROCESSING, 2021, 15 (03) : 493 - 500
  • [36] MULTI-SCALE TRANSFORMER NETWORK FOR SALIENCY PREDICTION ON 360-DEGREE IMAGES
    Lin, Xu
    Qing, Chunmei
    Tan, Junpeng
    Xu, Xiangmin
    2023 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP, 2023, : 1700 - 1704
  • [37] Pyramid Feature Attention Network for Saliency detection
    Zhao, Ting
    Wu, Xiangqian
    2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 3080 - 3089
  • [38] Global attention network for collaborative saliency detection
    Li, Ce
    Xuan, Shuxing
    Liu, Fenghua
    Chang, Enbing
    Wu, Hailei
    INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2023, 14 (02) : 407 - 417
  • [39] Global attention network for collaborative saliency detection
    Ce Li
    Shuxing Xuan
    Fenghua Liu
    Enbing Chang
    Hailei Wu
    International Journal of Machine Learning and Cybernetics, 2023, 14 : 407 - 417
  • [40] Toolbox and dataset for the development of saliency and scanpath models for omnidirectional/360° still images
    Gutierrez, Jesus
    David, Erwan
    Rai, Yashas
    Le Callet, Patrick
    SIGNAL PROCESSING-IMAGE COMMUNICATION, 2018, 69 : 35 - 42