Generating a Fusion Image: One's Identity and Another's Shape

被引:34
作者
Joo, Donggyu [1 ]
Kim, Doyeon [1 ]
Kim, Junmo [1 ]
机构
[1] Korea Adv Inst Sci & Technol, Sch Elect Engn, Daejeon, South Korea
来源
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) | 2018年
关键词
D O I
10.1109/CVPR.2018.00176
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Generating a novel image by manipulating two input images is an interesting research problem in the study of generative adversarial networks (GANs). We propose a new GAN-based network that generates a fusion image with the identity of input image x and the shape of input image y. Our network can simultaneously train on more than two image datasets in an unsupervised manner. We define an identity loss LI to catch the identity of image x and a shape loss LS to get the shape of y. In addition, we propose a novel training method called Min-Patch training to focus the generator on crucial parts of an image, rather than its entirety. We show qualitative results on the VGG Youtube Pose dataset, Eye dataset (MPIIGaze and UnityEyes), and the Photo-Sketch-Cartoon dataset.
引用
收藏
页码:1635 / 1643
页数:9
相关论文
共 24 条
[1]  
[Anonymous], 1986, Information processing in dynamical systems: Foundations of harmony theory
[2]  
[Anonymous], 2015, Advances in Neural Information Processing Systems
[3]   Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields [J].
Cao, Zhe ;
Simon, Tomas ;
Wei, Shih-En ;
Sheikh, Yaser .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :1302-1310
[4]   Personalizing Human Video Pose Estimation [J].
Charles, James ;
Pfister, Tomas ;
Magee, Derek ;
Hogg, David ;
Zisserman, Andrew .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :3063-3072
[5]  
Chen X., 2016, Advances in neural information processing systems, V2016, P2172
[6]   Generative Adversarial Networks [J].
Goodfellow, Ian ;
Pouget-Abadie, Jean ;
Mirza, Mehdi ;
Xu, Bing ;
Warde-Farley, David ;
Ozair, Sherjil ;
Courville, Aaron ;
Bengio, Yoshua .
COMMUNICATIONS OF THE ACM, 2020, 63 (11) :139-144
[7]  
Isola P, 2017, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, P1125, DOI [DOI 10.1109/CVPR.2017.632, 10.1109/CVPR.2017.632]
[8]  
Kim T., 2017, P 34 INT C MACH LEAR, P1857, DOI [DOI 10.1109/WPT.2017.7953894, 10.48550/arXiv.1703.05192]
[9]  
King DB, 2015, ACS SYM SER, V1214, P1
[10]   Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network [J].
Ledig, Christian ;
Theis, Lucas ;
Huszar, Ferenc ;
Caballero, Jose ;
Cunningham, Andrew ;
Acosta, Alejandro ;
Aitken, Andrew ;
Tejani, Alykhan ;
Totz, Johannes ;
Wang, Zehan ;
Shi, Wenzhe .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :105-114