PCMG:3D point cloud human motion generation based on self-attention and transformer

被引:4
作者
Ma, Weizhao [1 ]
Yin, Mengxiao [1 ,2 ]
Li, Guiqing [3 ]
Yang, Feng [1 ,2 ]
Chang, Kan [1 ,2 ]
机构
[1] Guangxi Univ, Sch Comp Elect & Informat, 100 East Univ Rd, Nanning 530004, Peoples R China
[2] Guangxi Univ, Guangxi Key Lab Multimedia Commun Network Technol, 100 East Univ Rd, Nanning 530004, Peoples R China
[3] South China Univ Technol, Sch Comp Sci & Engn, 381 Wushan Rd, Guangzhou 510006, Guangdong, Peoples R China
基金
中国国家自然科学基金;
关键词
Point cloud sequence generation; Conditional human motion generation; Transformer; Variational autoencoder;
D O I
10.1007/s00371-023-03063-x
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Previous methods for human motion generation have predominantly relied on skeleton representations to depict human poses and motion. These methods typically use a series of skeletons to represent the motion of a human. However, they are not directly suitable for handling the 3D point cloud sequences obtained from optical motion capture. To address this limitation, we propose a novel network called point cloud motion generation (PCMG) that can handle both skeleton-based motion representation and point cloud data from the human surface. PCMG is trained on finite point cloud sequences and is capable of generating infinite new point cloud sequences. By providing a predefined action label and shape label as input, PCMG generates a point cloud sequence that captures the semantics associated with these labels. PCMG achieves comparable results to state-of-the-art methods for action-conditional human motion generation, while outperforming previous approaches in terms of generation efficiency. The code for PCMG will be available at https://github.com/gxucg/PCMG
引用
收藏
页码:3765 / 3780
页数:16
相关论文
共 61 条
[1]  
Ahn H, 2018, IEEE INT CONF ROBOT, P5915
[2]   Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning [J].
Bhattacharya, Uttaran ;
Childs, Elizabeth ;
Rewkowski, Nicholas ;
Manocha, Dinesh .
PROCEEDINGS OF THE 29TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2021, 2021, :2027-2036
[3]   Implicit Neural Representations for Variable Length Human Motion Generation [J].
Cervantes, Pablo ;
Sekikawa, Yusuke ;
Sato, Ikuro ;
Shinoda, Koichi .
COMPUTER VISION - ECCV 2022, PT XVII, 2022, 13677 :356-372
[4]   DeepMoCap: Deep Optical Motion Capture Using Multiple Depth Sensors and Retro-Reflectors [J].
Chatzitofis, Anargyros ;
Zarpalas, Dimitrios ;
Kollias, Stefanos ;
Daras, Petros .
SENSORS, 2019, 19 (02)
[5]   MoCap-Solver: A Neural Solver for Optical Motion Capture Data [J].
Chen, Kang ;
Wang, Yupan ;
Zhang, Song-Hai ;
Xu, Sen-Zhe ;
Zhang, Weidong ;
Hu, Shi-Min .
ACM TRANSACTIONS ON GRAPHICS, 2021, 40 (04)
[6]   Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis [J].
Dai, Angela ;
Qi, Charles Ruizhongtai ;
Niessner, Matthias .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :6545-6554
[7]   Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos [J].
Fan, Hehe ;
Yang, Yi ;
Kankanhalli, Mohan .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :14199-14208
[8]  
Fang L., 2021, TRANSFORMER BASED CO, P2101
[9]   SOMA: Solving Optical Marker-Based MoCap Automatically [J].
Ghorbani, Nima ;
Black, Michael J. .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :11097-11106
[10]   Auto-labelling of Markers in Optical Motion Capture by Permutation Learning [J].
Ghorbani, Saeed ;
Etemad, Ali ;
Troje, Nikolaus F. .
ADVANCES IN COMPUTER GRAPHICS, CGI 2019, 2019, 11542 :167-178