Identity-aware CycleGAN for face photo-sketch synthesis and recognition

被引:67
作者
Fang, Yuke [1 ]
Deng, Weihong [1 ]
Du, Junping [1 ]
Hu, Jiani [1 ]
机构
[1] Beijing Univ Posts & Telecommun, Beijing 100876, Peoples R China
基金
中国国家自然科学基金; 国家重点研发计划;
关键词
Convolutional neural network; Generative adversarial network; Photo-sketch synthesis; Photo-sketch recognition; Identity-aware training;
D O I
10.1016/j.patcog.2020.107249
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Face photo-sketch synthesis and recognition has many applications in digital entertainment and law enforcement. Recently, generative adversarial networks (GANs) based methods have significantly improved the quality of image synthesis, but they have not explicitly considered the purpose of recognition. In this paper, we first propose an Identity-Aware CycleGAN (IACycleGAN) model that applies a new perceptual loss to supervise the image generation network. It improves CycleGAN on photo-sketch synthesis by paying more attention to the synthesis of key facial regions, such as eyes and nose, which are important for identity recognition. Furthermore, we develop a mutual optimization procedure between the synthesis model and the recognition model, which iteratively synthesizes better images by IACycleGAN and enhances the recognition model by the triplet loss of the generated and real samples. Extensive experiments are performed on both photo-to-sketch and sketch-to-photo tasks using the widely used CUFS and CUFSF databases. The results show that the proposed method performs better than several state-of-the-art methods in terms of both synthetic image quality and photo-sketch recognition accuracy. (C) 2020 Elsevier Ltd. All rights reserved.
引用
收藏
页数:11
相关论文
共 45 条
[1]   A Comparative Study on Face Sketch Synthesis [J].
Akram, Adeel ;
Wang, Nannan ;
Li, Jie ;
Gao, Xinbo .
IEEE ACCESS, 2018, 6 :37084-37093
[2]  
[Anonymous], ARXIV181210889 CORR
[3]  
[Anonymous], 2013, ARXIV E PRINTS
[4]  
[Anonymous], 1998, The AR Face Database Technical Report 24
[5]  
CVC
[6]  
[Anonymous], P 2 INT C AUD VID BA
[7]  
Bhatt H.S., 2012, Memetic approach for matching sketches with digital face images
[8]   A new LDA-based face recognition system which can solve the small sample size problem [J].
Chen, LF ;
Liao, HYM ;
Ko, MT ;
Lin, JC ;
Yu, GJ .
PATTERN RECOGNITION, 2000, 33 (10) :1713-1726
[9]   Face Sketch-Photo Synthesis and Retrieval Using Sparse Representation [J].
Gao, Xinbo ;
Wang, Nannan ;
Tao, Dacheng ;
Li, Xuelong .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2012, 22 (08) :1213-1226
[10]  
Goodfellow IJ, 2014, ADV NEUR IN, V27, P2672