Neural style transfer based on deep feature synthesis

Neural Style Transfer makes full use of the high-level features of deep neural networks, so stylized images can represent content and style features on high-level semantics. But neural networks are end-to-end black box systems. Previous style transfer models are based on the overall features of the image when constructing the target image, so they cannot effectively intervene in the content and style representations. This paper presents a locally controllable nonparametric neural style transfer model. We treat style transfer as a feature matching process independent of neural networks and propose a deep-to-shallow feature synthesis algorithm. The target feature map is synthesized layer by layer in the deep feature space and then transformed into the target image. Because the feature synthesis is a local manipulation on feature maps, it is easy to control the local texture structure, content details and texture distribution. Based on our synthesis algorithm, we propose a multi-exemplar synthesis method that can make local stroke directions better match content semantics or combine multiple styles into a single image. Our experiments show that our model can produce more impressive results than previous methods.