Saliency-guided image translation

被引：0
作者：
Jiang, Lai [1 ]
Dai, Ning [1 ]
Xu, Mai [1 ]
Deng, Xin [2 ]
Li, Shengxi [1 ]
机构：
[1] School of Electronic and Engineering, Beihang University, Beijing
[2] School of Cyber Science and Technology, Beihang University, Beijing
来源：
Beijing Hangkong Hangtian Daxue Xuebao/Journal of Beijing University of Aeronautics and Astronautics | 2023年 / 49卷 / 10期
基金：
中国国家自然科学基金;
关键词：
attention mechanism; dataset; generative adversarial network; image translation; saliency;
D O I：
10.13700/j.bh.1001-5965.2021.0732
中图分类号：
学科分类号：
摘要：
This paper proposes a novel task for saliency-guided image translation, with the goal of image-to-image translation conditioned on the user specified saliency map. To address this problem, we develop a novel generative adversarial network (GAN) method -based model, called SalG-GAN method. Given the original image and target saliency map, proposed method can generate a translated image that satisfies the target saliency map. In proposed method, a disentangled representation framework is proposed to encourage the model to learn diverse translations for the same target saliency condition. A saliency-based attention module is introduced as a special attention mechanism to facilitate the developed structures of saliency-guided generator, saliency cue encoder, and saliency-guided global and local discriminators. Furthermore, we build a synthetic dataset and a real-world dataset with labeled visual attention for training and evaluating proposed method. The experimental results on both datasets verify the effectiveness of our model for saliency-guided image translation. © 2023 Beijing University of Aeronautics and Astronautics (BUAA). All rights reserved.
引用
收藏
页码：2689 / 2698
页数：9
相关论文
共 37 条
  • [11] BAU D, ZHU J Y, STROBELT H, Et al., GAN dissection: Visualizing and understanding generative adversarial networks
  • [12] YU J H, LIN Z, YANG J M, Et al., Free-form image inpainting with gated convolution, 2019 IEEE/CVF International Conference on Computer Vision, pp. 4470-4479, (2020)
  • [13] MATEESCU V A, BAJIC I V., Visual attention retargeting, IEEE MultiMedia, 23, 1, pp. 82-91, (2016)
  • [14] MECHREZ R, SHECHTMAN E, ZELNIK-MANOR L., Saliency driven image manipulation, Machine Vision and Applications, 30, 2, pp. 189-202, (2019)
  • [15] FRIED O, SHECHTMAN E, GOLDMAN D B, Et al., Finding dis-tractors in images, 2015 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1703-1712, (2015)
  • [16] NGUYEN T V, NI B B, LIU H R, Et al., Image re-attentionizing, IEEE Transactions on Multimedia, 15, 8, pp. 1910-1919, (2013)
  • [17] ITTI L, KOCH C, NIEBUR E., A model of saliency-based visual attention for rapid scene analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 11, pp. 1254-1259, (1998)
  • [18] JIANG L, XU M, WANG X F, Et al., Saliency-guided image translation, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16504-16513, (2021)
  • [19] CHEN Y C, CHANG K J, TSAI Y H, Et al., Guide your eyes: Learning image manipulation under saliency guidance, British Machine Vision Conference, (2019)
  • [20] WONG L K, LOW K L., Saliency retargeting: An approach to enhance image aesthetics, 2011 IEEE Workshop on Applications of Computer Vision, pp. 73-80, (2011)
1234