Attentive Semantic and Perceptual Faces Completion Using Self-attention Generative Adversarial Networks

被引:7
作者
Liu, Xiaowei [1 ]
Li, Kenli [1 ]
Li, Keqin [2 ]
机构
[1] Hunan Univ, Coll Informat Sci & Engn, Changsha 410081, Peoples R China
[2] SUNY Coll New Paltz, Dept Comp Sci, New Paltz, NY 12561 USA
基金
国家重点研发计划; 中国国家自然科学基金;
关键词
Attention mechanism; Images completion; Non-local neural net; Semantics completion; IMAGE; ALGORITHM;
D O I
10.1007/s11063-019-10080-2
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We propose an approach based on self-attention generative adversarial networks to accomplish the task of image completion where completed images become globally and locally consistent. Using self-attention GANs with contextual and other constraints, the generator can draw realistic images, where fine details are generated in the damaged region and coordinated with the whole image semantically. To train the consistent generator, i.e. image completion network, we employ global and local discriminators where the global discriminator is responsible for evaluating the consistency of the entire image, while the local discriminator assesses the local consistency by analyzing local areas containing completed regions only. Last but not least, attentive recurrent neural block is introduced to obtain the attention map about the missing part in the image, which will help the subsequent completion network to fill contents better. By comparing the experimental results of different approaches on CelebA dataset, our method shows relatively good results.
引用
收藏
页码:211 / 229
页数:19
相关论文
共 51 条
[1]  
[Anonymous], 2012, P NIPS
[2]  
[Anonymous], 2015, P ADV NEUR INF PROC
[3]  
[Anonymous], 2015, NIPS 15 P 28 INT C N
[4]  
[Anonymous], 1997, Neural Computation
[5]  
[Anonymous], 2015, CoRR
[6]  
[Anonymous], 2014, Neural Information Processing Systems
[7]  
Ba J., 2014, Multiple object recognition with visual attention
[8]   PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing [J].
Barnes, Connelly ;
Shechtman, Eli ;
Finkelstein, Adam ;
Goldman, Dan B. .
ACM TRANSACTIONS ON GRAPHICS, 2009, 28 (03)
[9]   A non-local algorithm for image denoising [J].
Buades, A ;
Coll, B ;
Morel, JM .
2005 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOL 2, PROCEEDINGS, 2005, :60-65
[10]   Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks [J].
Cho, Kyunghyun ;
Courville, Aaron ;
Bengio, Yoshua .
IEEE TRANSACTIONS ON MULTIMEDIA, 2015, 17 (11) :1875-1886