Sliced Wasserstein Distance for Neural Style Transfer

Neural Style Transfer (NST) aims to render a content image with the style of another image in the feature space of a Convolution Neural Network (CNN). A fundamental concept of NST is to define the features extracted from a CNN as a distribution so that the style similarity can be computed by measuring the distance between distributions. Conceptually, Wasserstein Distance (WD) is ideal for measuring the distance between distributions as it theoretically guarantees the similarity of style distributions with the WD between them equaling 0. However, due to the high computation cost of WD, previous WD-based methods either oversimplify the style distribution or only use a lower bound of WD, therefore, losing the theoretical guarantee of WD. In this paper, we propose a new style loss based on Sliced Wasserstein Distance (SWD), which has a theoretical approximation guarantee. Besides, an adaptive sampling algorithm is also proposed to further improve the style transfer results. Experiment results show that the proposed method improves the similarity of style distributions, and such improvements result in visually better style transfer results. (c) 2021 Elsevier Ltd. All rights reserved.