Face Sketch-Photo Synthesis Method Based on Multi-residual Dynamic Fusion Generative Adversarial Networks

被引：0

作者：

Sun R. ^{[1
,2
]}

Sun Q. ^{[1
,2
]}

Shan X. ^{[1
,2
]}

Zhang X. ^{[1
]}

机构：

[1] School of Computer Science and Information Engineering, Hefei University of Technology, Hefei

[2] Anhui Key Laboratory of Industry Safety and Emergency Technology, Hefei University of Technology, Hefei

来源：

Moshi Shibie yu Rengong Zhineng/Pattern Recognition and Artificial Intelligence | 2022年 / 35卷 / 03期

基金：

中国国家自然科学基金;

关键词：

Deep Learning; Face Sketch-Photo Synthesis; Generative Adversarial Networks; Multi-residual Dynamic Fusion;

D O I：

10.16451/j.cnki.issn1003-6059.202203002

中图分类号：

学科分类号：

摘要：

Aiming at the low definition and blurry details in the current face sketch-photo synthesis methods, a face sketch-photo synthesis method based on multi-residual dynamic fusion generative adversarial network is proposed. Firstly, a multi-residual dynamic fusion network is designed. Features are extracted from different dense residual modules and then the residual learning is conducted. Then, the corresponding offsets are generated on the basis of the diverse residual features at different levels. The sampling coordinates of the convolution kernels in different locations are changed according to the offsets. Consequently, the network is focused on important feature adaptively, and geometric detail information and high-level semantic information are integrated effectively without gradual information dropping and redundant information interference. Moreover, a multi-scale perceptual loss is introduced to conduct perceptual comparison on the synthetic images of different resolutions for the regularization of synthetic images from coarse to fine. Experiments on Chinese University of Hong Kong face sketch dataset show that the proposed method produces high-definition images with full detail and consistent color and the synthesized image is closer to real face images. © 2022, Science Press. All right reserved.

引用

页码：207 / 222

页数：15

共 30 条

[1]

LIU Q S, TANG X O, JIN H L, Et al., A Nonlinear Approach for Face Sketch Synthesis and Recognition, Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, I, pp. 1005-1010, (2005)

[2]

WANG X G, TANG X O., Face Photo-Sketch Synthesis and Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 11, pp. 1955-1967, (2009)

[3]

CHANG L, ZHOU M Q, DENG X M, Et al., Face Sketch Synthesis via Multivariate Output Regression, Proc of the 14th International Conference on Human-Computer Interaction: Design and Development Approaches, pp. 555-561, (2011)

[4]

ZHANG S C, GAO X B, WANG N N, Et al., Face Sketch Synthesis via Sparse Representation-Based Greedy Search, IEEE Transactions on Image Processing, 24, 8, pp. 2466-2477, (2015)

[5]

ZHANG D Y, LIN L, CHEN T S, Et al., Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning, IEEE Transactions on Image Processing, 26, 1, pp. 328-339, (2017)

[6]

GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, Et al., Ge-nerative Adversarial Nets, Proc of the 27th International Confe-rence on Neural Information Processing Systems, II, pp. 2672-2680, (2014)

[7]

KINGMA D P, WELLING M., Auto-Encoding Variational Bayes

[8]

ISOLA P, ZHU J Y, ZHOU T H, Et al., Image-to-Image Translation with Conditional Adversarial Networks, Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition, pp. 5967-5976, (2017)

[9]

WANG T C, LIU M Y, ZHU J Y, Et al., High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs, Proc of the IEEE/CVF Conference on Computer Vision and Pattern Re-cognition, pp. 8798-8807, (2018)

[10]

ZHU J Y, PARK T, ISOLA P, Et al., Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks, Proc of the IEEE International Conference on Computer Vision, pp. 2242-2251, (2017)

← 1 2 3 →