BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis

被引:2
作者
Castillo, Angela [1 ]
Escobar, Maria [1 ]
Jeanneret, Guillaume [2 ]
Pumarola, Albert [3 ]
Arbelaez, Pablo [1 ]
Thabet, Ali [3 ]
Sanakoyeu, Artsiom [3 ]
机构
[1] Univ Los Andes, Ctr Res & Format Artificial Intelligence, Santiago, Chile
[2] Univ Caen Normandie, ENSICAEN, CNRS, Caen, France
[3] Meta AI, New York, NY USA
来源
2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS, ICCVW | 2023年
关键词
D O I
10.1109/ICCVW60793.2023.00456
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Mixed reality applications require tracking the user's full-body motion to enable an immersive experience. However, typical head-mounted devices can only track head and hand movements, leading to a limited reconstruction of full-body motion due to variability in lower body configurations. We propose BoDiffusion - a generative diffusion model for motion synthesis to tackle this under-constrained reconstruction problem. We present a time and space conditioning scheme that allows BoDiffusion to leverage sparse tracking inputs while generating smooth and realistic full-body motion sequences. To the best of our knowledge, this is the first approach that uses the reverse diffusion process to model full-body tracking as a conditional sequence generation task. We conduct experiments on the large-scale motion-capture dataset AMASS and show that our approach outperforms the state-of-the-art approaches by a significant margin in terms of full-body motion realism and joint reconstruction error.
引用
收藏
页码:4223 / 4233
页数:11
相关论文
共 60 条
[11]   MoGlow: Probabilistic and Controllable Motion Synthesis Using Normalising Flows [J].
Henter, Gustav Eje ;
Alexanderson, Simon ;
Beskow, Jonas .
ACM TRANSACTIONS ON GRAPHICS, 2020, 39 (06)
[12]  
Ho J., 2020, P NIPS, V33, P6840
[13]  
Ho Jonathan, 2022, ARXIV220403458
[14]   Phase-Functioned Neural Networks for Character Control [J].
Holden, Daniel ;
Komura, Taku ;
Saito, Jun .
ACM TRANSACTIONS ON GRAPHICS, 2017, 36 (04)
[15]  
Jiang Jiaxi., 2022, P EUR C COMP VIS
[16]  
Jiang Y., 2022, ARXIV220315720
[17]  
Jones Brennan, 2021, CSCW 2021
[18]  
Kaufmann M., 2021, PROC IEEECVF INT C C, P11510
[19]  
Kim Jihoon, 2022, ARXIV220900349
[20]  
King DB, 2015, ACS SYM SER, V1214, P1