High-fidelity 3D real-time facial animation using infrared structured light sensing system

被引：10
作者：
Ye, Yuping [1 ,2 ]
Song, Zhan [1 ,3 ]
Zhao, Juan [1 ,3 ]
机构：
[1] Chinese Acad Sci, Shenzhen Inst Adv Technol, Shenzhen, Peoples R China
[2] Univ Chinese Acad Sci, Beijing, Peoples R China
[3] Chinese Univ Hong Kong, Hong Kong, Peoples R China
来源：
COMPUTERS & GRAPHICS-UK | 2022年 / 104卷
关键词：
Structured light; Facial animation; Non-rigid registration;
D O I：
10.1016/j.cag.2022.03.007
中图分类号：
TP31 [计算机软件];
学科分类号：
081202 ; 0835 ;
摘要：
The acquisition of 3D facial models is crucial in the gaming and film industries. In this study, we developed a facial acquisition system based on infrared structured light sensors to obtain facialexpression models with high fidelity and accuracy. First, we employed time-multiplexing structured light to obtain an accurate and dense point cloud. The template model was then warped to the captured facial expression. A model-tracking method based on optical flow was applied to track the registered 3D model displacement. Finally, the live 3D mesh was textured using high-resolution images captured by three color cameras. Experiments were conducted on human faces to demonstrate the performance of the proposed system and methods. (c) 2022 Elsevier Ltd. All rights reserved.
引用
收藏
页码：46 / 58
页数：13
相关论文
共 67 条
[21]  
Grunnet-Jepsen A., 2018, Projectors for Intel RealSense depth cameras d4xx
[22]   The Relightables: Volumetric Performance Capture of Humans with Realistic Relighting [J].
Guo, Kaiwen ;
Lincoln, Peter ;
Davidson, Philip ;
Busch, Jay ;
Yu, Xueming ;
Whalen, Matt ;
Harvey, Geoff ;
Orts-Escolano, Sergio ;
Pandey, Rohit ;
Dourgarian, Jason ;
Tang, Danhang ;
Tkach, Anastasia ;
Kowdle, Adarsh ;
Cooper, Emily ;
Dou, Mingsong ;
Fanello, Sean ;
Fyffe, Graham ;
Rhemann, Christoph ;
Taylor, Jonathan ;
Debevec, Paul ;
Izadi, Shahram .
ACM TRANSACTIONS ON GRAPHICS, 2019, 38 (06)
[23]   FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks [J].
Ilg, Eddy ;
Mayer, Nikolaus ;
Saikia, Tonmoy ;
Keuper, Margret ;
Dosovitskiy, Alexey ;
Brox, Thomas .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :1647-1655
[24]   A Style-Based Generator Architecture for Generative Adversarial Networks [J].
Karras, Tero ;
Laine, Samuli ;
Aila, Timo .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :4396-4405
[25]  
King DE, 2009, J MACH LEARN RES, V10, P1755
[26]   DeepFaceFlow: In-the-wild Dense 3D Facial Motion Estimation [J].
Koujan, Mohammad Rami ;
Roussos, Anastasios ;
Zafeiriou, Stefanos .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :6617-6626
[27]   AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild" [J].
Lattas, Alexandros ;
Moschoglou, Stylianos ;
Gecer, Baris ;
Ploumpis, Stylianos ;
Triantafyllou, Vasileios ;
Ghosh, Abhijeet ;
Zafeiriou, Stefanos .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :757-766
[28]  
Li F, 2012, 2012 IEEE FIFTH INTERNATIONAL CONFERENCE ON ADVANCED COMPUTATIONAL INTELLIGENCE (ICACI), P662, DOI 10.1109/ICACI.2012.6463249
[29]   Realtime Facial Animation with On-the-fly Correctives [J].
Li, Hao ;
Yu, Jihun ;
Ye, Yuting ;
Bregler, Chris .
ACM TRANSACTIONS ON GRAPHICS, 2013, 32 (04)
[30]   Lightweight wrinkle synthesis for 3D facial modeling and animation [J].
Li, Jun ;
Xu, Weiwei ;
Cheng, Zhiquan ;
Xu, Kai ;
Klein, Reinhard .
COMPUTER-AIDED DESIGN, 2015, 58 :117-122
1234567