High-Quality Streamable Free-Viewpoint Video

被引：432

作者：

Collet, Alvaro ^{[1
]}

Chuang, Ming ^{[1
]}

Sweeney, Pat ^{[1
]}

Gillett, Don ^{[1
]}

Evseev, Dennis ^{[1
]}

Calabrese, David ^{[1
]}

Hoppe, Hugues ^{[1
]}

Kirk, Adam ^{[1
]}

Sullivan, Steve ^{[1
]}

机构：

[1] Microsoft Corp, Redmond, WA 98052 USA

来源：

ACM TRANSACTIONS ON GRAPHICS | 2015年 / 34卷 / 04期

关键词：

multi-view stereo; surface reconstruction; 3D video; mesh tracking; geometry compression; MPEG;

D O I：

10.1145/2766945

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

We present the first end-to-end solution to create high-quality free-viewpoint video encoded as a compact data stream. Our system records performances using a dense set of RGB and IR video cameras, generates dynamic textured surfaces, and compresses these to a streamable 3D video format. Four technical advances contribute to high fidelity and robustness: multimodal multi-view stereo fusing RGB, IR, and silhouette information; adaptive meshing guided by automatic detection of perceptually salient areas; mesh tracking to create temporally coherent subsequences; and encoding of tracked textured meshes as an MPEG video stream. Quantitative experiments demonstrate geometric accuracy, texture fidelity, and encoding efficiency. We release several datasets with calibrated inputs and processed results to foster future research.

引用

页数：13