High-fidelity 3D real-time facial animation using infrared structured light sensing system

被引：8
作者：
Ye, Yuping [1 ,2 ]
Song, Zhan [1 ,3 ]
Zhao, Juan [1 ,3 ]
机构：
[1] Chinese Acad Sci, Shenzhen Inst Adv Technol, Shenzhen, Peoples R China
[2] Univ Chinese Acad Sci, Beijing, Peoples R China
[3] Chinese Univ Hong Kong, Hong Kong, Peoples R China
来源：
COMPUTERS & GRAPHICS-UK | 2022年 / 104卷
关键词：
Structured light; Facial animation; Non-rigid registration;
D O I：
10.1016/j.cag.2022.03.007
中图分类号：
TP31 [计算机软件];
学科分类号：
081202 ; 0835 ;
摘要：
The acquisition of 3D facial models is crucial in the gaming and film industries. In this study, we developed a facial acquisition system based on infrared structured light sensors to obtain facialexpression models with high fidelity and accuracy. First, we employed time-multiplexing structured light to obtain an accurate and dense point cloud. The template model was then warped to the captured facial expression. A model-tracking method based on optical flow was applied to track the registered 3D model displacement. Finally, the live 3D mesh was textured using high-resolution images captured by three color cameras. Experiments were conducted on human faces to demonstrate the performance of the proposed system and methods. (c) 2022 Elsevier Ltd. All rights reserved.
引用
收藏
页码：46 / 58
页数：13
相关论文
共 67 条
[1]  
Achenbach J, 2015, VISION MODELING VISU
[2]  
Alexander O., 2009, ACM SIGGRAPH 2009 CO, p12: 1
[3]   The Digital Emily Project: Achieving a Photorealistic Digital Actor [J].
Alexander, Oleg ;
Rogers, Mike ;
Lambeth, William ;
Chiang, Jen-Yuan ;
Ma, Wan-Chun ;
Wang, Chuan-Chang ;
Debevec, Paul .
IEEE COMPUTER GRAPHICS AND APPLICATIONS, 2010, 30 (04) :20-31
[4]   The space of human body shapes: reconstruction and parameterization from range scans [J].
Allen, B ;
Curless, B ;
Popovic, Z .
ACM TRANSACTIONS ON GRAPHICS, 2003, 22 (03) :587-594
[5]  
Amberg Brian, 2007, CVPR '07. IEEE Conference on Computer Vision and Pattern Recognition, P1
[6]  
Baumberg A., 2002, Bmvc, V3, P5, DOI DOI 10.5244/C.16.38
[7]   High-Quality Single-Shot Capture of Facial Geometry [J].
Beeler, Thabo ;
Bickel, Bernd ;
Beardsley, Paul ;
Sumner, Bob ;
Gross, Markus .
ACM TRANSACTIONS ON GRAPHICS, 2010, 29 (04)
[8]   A morphable model for the synthesis of 3D faces [J].
Blanz, V ;
Vetter, T .
SIGGRAPH 99 CONFERENCE PROCEEDINGS, 1999, :187-194
[9]   Online Modeling For Realtime Facial Animation [J].
Bouaziz, Sofien ;
Wang, Yangang ;
Pauly, Mark .
ACM TRANSACTIONS ON GRAPHICS, 2013, 32 (04)
[10]   Real-time 3D Neural Facial Animation from Binocular Video [J].
Cao, Chen ;
Agrawal, Vasu ;
De la Torre, Fernando ;
Chen, Lele ;
Saragih, Jason ;
Simon, Tomas ;
Sheikh, Yaser .
ACM TRANSACTIONS ON GRAPHICS, 2021, 40 (04)
1234567