Real-Time High-Fidelity Facial Performance Capture

被引:170
作者
Cao, Chen [1 ,2 ]
Bradley, Derek [2 ]
Zhou, Kun [1 ]
Beeler, Thabo [2 ]
机构
[1] Zhejiang Univ, State Key Lab CAD&CG, Hangzhou, Zhejiang, Peoples R China
[2] Disney Res Zurich, Zurich, Switzerland
来源
ACM TRANSACTIONS ON GRAPHICS | 2015年 / 34卷 / 04期
关键词
Real-time face reconstruction; local wrinkle model; high-fidelity performance capture; GEOMETRY;
D O I
10.1145/2766943
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
We present the first real-time high-fidelity facial capture method. The core idea is to enhance a global real-time face tracker, which provides a low-resolution face mesh, with local regressors that add in medium-scale details, such as expression wrinkles. Our main observation is that although wrinkles appear in different scales and at different locations on the face, they are locally very self-similar and their visual appearance is a direct consequence of their local shape. We therefore train local regressors from high-resolution capture data in order to predict the local geometry from local appearance at runtime. We propose an automatic way to detect and align the local patches required to train the regressors and run them efficiently in real-time. Our formulation is particularly designed to enhance the low-resolution global tracker with exactly the missing expression frequencies, avoiding superimposing spatial frequencies in the result. Our system is generic and can be applied to any real-time tracker that uses a global prior, e.g. blend-shapes. Once trained, our online capture approach can be applied to any new user without additional training, resulting in high-fidelity facial performance reconstruction with person-specific wrinkle details from a monocular video camera in real-time.
引用
收藏
页数:9
相关论文
共 31 条
  • [1] [Anonymous], SCA
  • [2] [Anonymous], NAT COMMUN, DOI DOI 10.1038/NCOMMS2680
  • [3] [Anonymous], 2009, CVPR
  • [4] [Anonymous], ICCV
  • [5] [Anonymous], 2007, P 18 EUROGRAPHICS C, DOI [10.2312/EGWR/EGSR07/183-194., DOI 10.2312/EGWR/EGSR07/183-194.6]
  • [6] [Anonymous], 2011, lggepflch, DOI [DOI 10.1145/1964921.1964972, 10.1145/2010324.1964972]
  • [7] [Anonymous], ECCV
  • [8] [Anonymous], ACM T GRAPH IN PRESS
  • [9] Beeler T, 2012, LECT NOTES COMPUT SC, V7572, P30, DOI 10.1007/978-3-642-33718-5_3
  • [10] Facial Performance Enhancement Using Dynamic Shape Space Analysis
    Bermano, Amit H.
    Bradley, Derek
    Beeler, Thabo
    Zund, Fabio
    Nowrouzezahrai, Derek
    Baran, Ilya
    Sorkine-Hornung, Olga
    Pfister, Hanspeter
    Sumner, Robert W.
    Bickel, Bernd
    Gross, Markus
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2014, 33 (02):