MFMAM: Image inpainting via multi-scale feature module with attention module

被引:61
|
作者
Chen, Yuantao [1 ,2 ]
Xia, Runlong [3 ]
Yang, Kai [4 ]
Zou, Ke [5 ]
机构
[1] Hunan Univ Informat Technol, Sch Comp Sci & Engn, Changsha, Hunan, Peoples R China
[2] Hunan Univ Informat Technol, Hunan Prov Higher Educ Key Lab Intelligent Sensing, Changsha, Hunan, Peoples R China
[3] Mt Yuelu Breeding Innovat Ctr Ltd, Changsha, Hunan, Peoples R China
[4] Hunan ZOOMLION Intelligent Technol Corp Ltd, Changsha, Hunan, Peoples R China
[5] Hunan Initial New Mat Corp Ltd, Loudi, Hunan, Peoples R China
关键词
Image inpainting; Deep learning; Multi-scale feature; Deep level features; Attention module;
D O I
10.1016/j.cviu.2023.103883
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Currently, most image inpainting algorithms based on deep learning cause information loss when acquiring deep level features. It is not conducive to the image inpainting of texture details and often ignore the image restoration of semantic features, which generates image inpainting results with unreasonable structures. To address those problems, we have proposed improved image inpainting network using multi-scale feature module and improved attention module. Firstly, we propose the multi-scale fusion module based on dilated convolution to reduce information loss during the convolution process by fusing multi-scale features when acquiring image deep level features. Then, the attention module can strengthen the ability of image semantic inpainting and ensure that the proposed model can generate clear texture inpainting results. To ensure the consistency of image inpainting details and styles, the style loss and perceptual loss functions are introduced in the proposed network. The qualitative experimental results on CelebA-HQ, Places2 and Outdoor Scene datasets and common evaluation metrics such as PSNR, SSIM and FID. These metrics can validate the proposed method to be superior to the state-of-the-arts.
引用
收藏
页数:11
相关论文
共 50 条
  • [1] Image Inpainting Using Multi-Scale Feature Joint Attention Model
    Lin X.
    Zhou Y.
    Li D.
    Huang W.
    Sheng B.
    Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics, 2022, 34 (08): : 1260 - 1271
  • [2] Multi-scale attention network for image inpainting
    Qin, Jia
    Bai, Huihui
    Zhao, Yao
    COMPUTER VISION AND IMAGE UNDERSTANDING, 2021, 204
  • [3] Image Inpainting Based on Structural Constraint and Multi-Scale Feature Fusion
    Fan, Yao
    Shi, Yingnan
    Zhang, Ningjun
    Chu, Yanli
    IEEE ACCESS, 2023, 11 : 16567 - 16587
  • [4] An improved Image Inpainting Algorithm Based on Attention Fusion Module
    Ding, Zhen
    Wang, Tong
    Haul, Kuangrong
    2022 34TH CHINESE CONTROL AND DECISION CONFERENCE, CCDC, 2022, : 5087 - 5092
  • [5] Spatially adaptive multi-scale contextual attention for image inpainting
    Xueting Wang
    Yiyan Chen
    Toshihiko Yamasaki
    Multimedia Tools and Applications, 2022, 81 : 31831 - 31846
  • [6] Image Inpainting Based Multi-scale Gated Convolution and Attention
    Jiang, Hualiang
    Ma, Xiaohu
    Yang, Dongdong
    Zhao, Jiaxin
    Shen, Yao
    ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2022, PT II, 2022, 13530 : 407 - 418
  • [7] Spatially adaptive multi-scale contextual attention for image inpainting
    Wang, Xueting
    Chen, Yiyan
    Yamasaki, Toshihiko
    MULTIMEDIA TOOLS AND APPLICATIONS, 2022, 81 (22) : 31831 - 31846
  • [8] Image inpainting via Multi-scale Adaptive Priors
    Wang, Yufeng
    Guo, Dongsheng
    Zhao, Haoru
    Yang, Min
    Zheng, Haiyong
    PATTERN RECOGNITION, 2025, 162
  • [9] ROTATING TARGET DETECTION OF SAR IMAGE BASED ON MULTI-SCALE ATTENTION MODULE FOR INSHORE SHIPS
    Liu, Haodong
    Wang, Lu
    Zhao, Chunhui
    Wang, Ning
    Chen, Jikang
    IGARSS 2023 - 2023 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM, 2023, : 7034 - 7037
  • [10] ATM-DEN: Image Inpainting via attention transfer module and Decoder-Encoder network
    Zhang, Siwei
    Chen, Yuantao
    SIGNAL PROCESSING-IMAGE COMMUNICATION, 2025, 133