MoveBox: Democratizing MoCap for the Microsoft Rocketbox Avatar Library

被引：24

作者：

Gonzalez-Franco, Mar ^{[1
]}

Egan, Zelia ^{[1
]}

Peachey, Matthew ^{[2
]}

Antley, Angus ^{[1
]}

Randhavane, Tanmay ^{[1
]}

Panda, Payod ^{[1
]}

Zhang, Yaying ^{[1
]}

Wang, Cheng Yao ^{[4
]}

Reilly, Derek F. ^{[2
]}

Peck, Tabitha C. ^{[3
]}

Won, Andrea Stevenson ^{[4
]}

Steed, Anthony ^{[5
]}

Ofek, Eyal ^{[1
]}

机构：

[1] Microsoft Res, Redmond, WA 98052 USA

[2] Dalhousie Univ, Halifax, NS, Canada

[3] Davidson Coll, Davidson, NC 28036 USA

[4] Cornell Univ, Ithaca, NY USA

[5] UCL, London, England

来源：

2020 IEEE INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND VIRTUAL REALITY (AIVR 2020) | 2020年

关键词：

Avatars; Animation; Motion Capture; MoCap; Rigging; Depth Cameras; Lip Sync; REPRESENTATION; EMBODIMENT; ILLUSION;

D O I：

10.1109/AIVR50618.2020.00026

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

This paper presents MoveBox an open sourced toolbox for animating motion captured (MoCap) movements onto the Microsoft Rocketbox library of avatars. Motion capture is performed using a single depth sensor, such as Azure Kinect or Windows Kinect V2. Motion capture is performed in real-time using a single depth sensor, such as Azure Kinect or Windows Kinect V2, or extracted from existing RGB videos offline leveraging deep-learning computer vision techniques. Our toolbox enables real-time animation of the user's avatar by converting the transformations between systems that have different joints and hierarchies. Additional features of the toolbox include recording, playback and looping animations, as well as basic audio lip sync, blinking and resizing of avatars as well as finger and hand animations. Our main contribution is both in the creation of this open source tool as well as the validation on different devices and discussion of MoveBox's capabilities by end users.

引用

页码：91 / 98

页数：8

共 45 条

[1] I'm a Giant: Walking in Large Virtual Environments at High Speed Gains [J].

Abtahi, Parastoo ;

Gonzalez-Franco, Mar ;

Ofek, Eyal ;

Steed, Anthony .

CHI 2019: PROCEEDINGS OF THE 2019 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, 2019,

[2] MeCap: Whole-Body Digitization for Low-Cost VR/AR Headsets [J].

Ahuja, Karan ;

Harrison, Chris ;

Goel, Mayank ;

Xiao, Robert .

PROCEEDINGS OF THE 32ND ANNUAL ACM SYMPOSIUM ON USER INTERFACE SOFTWARE AND TECHNOLOGY (UIST 2019), 2019, :453-462

[3]

[Anonymous], 2020, OCULUS LIPSYNC UNITY

[4]

[Anonymous], 2013, P 19 ACM S VIRT REAL, DOI [10.1145/2503713.2503747, DOI 10.1145/2503713.2503747]

[5]

[Anonymous], 2020, MOTIONSHADOW

[6]

[Anonymous], 2020, SALSA LIP SYNC EMOTE

[7]

[Anonymous], 2020, MICROSOFT ROCKETBOX

[8]

Bailenson JN, 2006, COMP SUPP COMP W SER, V34, P1

[9] High-Quality Passive Facial Performance Capture using Anchor Frames [J].

Beeler, Thabo ;

Hahn, Fabian ;

Bradley, Derek ;

Bickel, Bernd ;

Beardsley, Paul ;

Gotsman, Craig ;

Sumner, Robert W. ;

Gross, Markus .

ACM TRANSACTIONS ON GRAPHICS, 2011, 30 (04)

[10] OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields [J].

Cao, Zhe ;

Hidalgo, Gines ;

Simon, Tomas ;

Wei, Shih-En ;

Sheikh, Yaser .

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2021, 43 (01) :172-186

← 1 2 3 4 5 →