Stylizing Video by Example

被引:62
作者
Jamriska, Ondrej [1 ]
Sochorova, Sarka [1 ]
Texler, Ondrej [1 ]
Lukac, Michal [2 ]
Fiser, Jakub [2 ]
Lu, Jingwan [2 ]
Shechtman, Eli [2 ]
Sykora, Daniel [1 ]
机构
[1] Czech Tech Univ, Fac Elect Engn, Prague, Czech Republic
[2] Adobe Res, San Francisco, CA USA
来源
ACM TRANSACTIONS ON GRAPHICS | 2019年 / 38卷 / 04期
关键词
style transfer;
D O I
10.1145/3306346.3323006
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
We introduce a new example-based approach to video stylization, with a focus on preserving the visual quality of the style, user controllability and applicability to arbitrary video. Our method gets as input one or more keyframes that the artist chooses to stylize with standard painting tools. It then automatically propagates the stylization to the rest of the sequence. To facilitate this while preserving visual quality, we developed a new type of guidance for state-of-art patch-based synthesis, that can be applied to any type of video content and does not require any additional information besides the video itself and a user-specified mask of the region to be stylized. We further show a temporal blending approach for interpolating style between keyframes that preserves texture coherence, contrast and high frequency details. We evaluate our method on various scenes from real production setting and provide a thorough comparison with prior art.
引用
收藏
页数:11
相关论文
共 66 条
[11]  
[Anonymous], 2015, COMPUTER GRAPHICS FO
[12]   PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing [J].
Barnes, Connelly ;
Shechtman, Eli ;
Finkelstein, Adam ;
Goldman, Dan B. .
ACM TRANSACTIONS ON GRAPHICS, 2009, 28 (03)
[13]   A Dynamic Noise Primitive for Coherent Stylization [J].
Benard, P. ;
Lagae, A. ;
Vangorp, P. ;
Lefebvre, S. ;
Drettakis, G. ;
Thollot, J. .
COMPUTER GRAPHICS FORUM, 2010, 29 (04) :1497-1506
[14]   Stylizing Animation By Example [J].
Benard, Pierre ;
Cole, Forrester ;
Kass, Michael ;
Mordatch, Igor ;
Hegarty, James ;
Senn, Martin Sebastian ;
Fleischer, Kurt ;
Pesare, Davide ;
Breeden, Katherine .
ACM TRANSACTIONS ON GRAPHICS, 2013, 32 (04)
[15]  
Bhat P, 2008, LECT NOTES COMPUT SC, V5303, P114, DOI 10.1007/978-3-540-88688-4_9
[16]   An L1 Image Transform for Edge-Preserving Smoothing and Scene-Level Intrinsic Decomposition [J].
Bi, Sai ;
Han, Xiaoguang ;
Yu, Yizhou .
ACM TRANSACTIONS ON GRAPHICS, 2015, 34 (04)
[17]  
BOUSSEAU A., 2006, Proceedings of International Symposium on Non-Photorealistic Animation and Rendering, P141
[18]  
Bousseau A, 2007, ACM T GRAPHIC, V26, DOI [10.1145/1239451.1239555, 10.1145/1276377.1276507]
[19]  
Browning Mark., 2014, Proceedings of the Workshop on Non-Photorealistic Animation and Rendering, NPAR'14, P63
[20]  
Chang CH, 2018, IEEE IMAGE PROC, P3179, DOI 10.1109/ICIP.2018.8451072