EDBGAN: Image Inpainting via an Edge-Aware Dual Branch Generative Adversarial Network

被引:10
|
作者
Chen, Minyu [1 ,2 ]
Liu, Zhi [1 ,2 ]
机构
[1] Shanghai Univ, Shanghai Inst Adv Commun & Data Sci, Shanghai 200444, Peoples R China
[2] Shanghai Univ, Sch Commun & Informat Engn, Shanghai 200444, Peoples R China
基金
中国国家自然科学基金;
关键词
Feature extraction; Image edge detection; Convolution; Generators; Task analysis; Logic gates; Generative adversarial networks; Attention mechanism; dual branch encoder-decoder; generative adversarial network; image inpainting;
D O I
10.1109/LSP.2021.3070738
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
As deep learning technology develops rapidly, image inpainting methods have made significant progress in generating reasonable contents for images with large and irregular holes. Nevertheless, existing methods either use one encoder-decoder to generate results in one step without making good use of structure features which are helpful, or adopt two encoder-decoders to recover structures and textures subsequently where the second encoder-decoder used for recovering textures relies heavily on the first encoder-decoder. Thus, an one-stage method which simultaneously utilizes structures and textures is promising. In this letter, we propose a dual branch encoder-decoder, whose texture branch and edge branch can simultaneously extract features from the masked RGB image and its corresponding edge map. Moreover, we propose a lightweight mutually guided attention block (MGAB), which makes features of two branches guide each other from low level to high level. At the end of two branches, we apply dual attention block (DAB) to perform feature fusion, which uses self-attention mechanism and lightweight attention mechanism on features of texture branch and edge branch, respectively. Extensive experiments demonstrate that our proposed method is effective in recovering edges and textures and achieves the state-of-the-art performance.
引用
收藏
页码:842 / 846
页数:5
相关论文
共 50 条
  • [1] Edge-Aware Image Super-Resolution Using a Generative Adversarial Network
    Das B.
    Roy S.D.
    SN Computer Science, 4 (2)
  • [2] Edge-aware image outpainting with attentional generative adversarial networks
    Li, Xiaoming
    Zhang, Hengzhi
    Feng, Lei
    Hu, Jing
    Zhang, Rongguo
    Qiao, Qiang
    IET IMAGE PROCESSING, 2022, 16 (07) : 1807 - 1821
  • [3] Image Inpainting via Enhanced Generative Adversarial Network
    Wang, Qiang
    Fan, Huijie
    2021 27TH INTERNATIONAL CONFERENCE ON MECHATRONICS AND MACHINE VISION IN PRACTICE (M2VIP), 2021,
  • [4] EDGE-AWARE CONTEXT ENCODER FOR IMAGE INPAINTING
    Liao, Liang
    Hu, Ruimin
    Xiao, Jing
    Wang, Zhongyuan
    2018 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2018, : 3156 - 3160
  • [5] Edge-Guided Generative Adversarial Network for Image Inpainting
    Xu, Shunxin
    Liu, Dong
    Xiong, Zhiwei
    2017 IEEE VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2017,
  • [6] An Edge-Aware Transformer Framework for Image Inpainting Detection
    Hu, Liangpei
    Li, Yuanman
    You, Jiaxiang
    Liang, Rongqin
    Li, Xia
    ARTIFICIAL INTELLIGENCE AND SECURITY, ICAIS 2022, PT II, 2022, 13339 : 648 - 660
  • [7] POLARIMETRIC SAR IMAGE CLASSIFICATION BASED ON EDGE-AWARE DUAL BRANCH FULLY CONVOLUTIONAL NETWORK
    Gao, Feng
    Chen, Yanqiao
    Chai, Xinghua
    Wu, Bin
    Peng, Cheng
    Xing, Ruoting
    Li, Yangyang
    International Geoscience and Remote Sensing Symposium (IGARSS), 2021, : 4728 - 4731
  • [8] Generative image inpainting via edge structure and color aware fusion
    Shao, Hang
    Wang, Yongxiong
    Fu, Yinghua
    Yin, Zhong
    SIGNAL PROCESSING-IMAGE COMMUNICATION, 2020, 87 (87)
  • [9] Adversarial Edge-Aware Image Colorization With Semantic Segmentation
    Kong, Guangqian
    Tian, Huan
    Duan, Xun
    Long, Huiyun
    IEEE ACCESS, 2021, 9 (09): : 28194 - 28203
  • [10] A Novel Dual U-Net Generative Adversarial Network for Image Inpainting
    Yuan, Jianjun
    Wu, Hong
    Wu, Fujun
    INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE, 2024, 38 (06)