Neural Rerendering in the Wild

被引：125

作者：

Meshry, Moustafa ^{[1
,3
]}

Goldman, Dan B. ^{[2
]}

Khamis, Sameh ^{[2
]}

Hoppe, Hugues ^{[2
]}

Pandey, Rohit ^{[2
]}

Snavely, Noah ^{[2
]}

Martin-Brualla, Ricardo ^{[2
]}

机构：

[1] Univ Maryland, College Pk, MD 20742 USA

[2] Google Inc, Mountain View, CA USA

[3] Google, Mountain View, CA 94043 USA

来源：

2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019) | 2019年

关键词：

D O I：

10.1109/CVPR.2019.00704

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We explore total scene capture - recording, modeling, and rerendering a scene under varying appearance such as season and time of day. Starting from internet photos of a tourist landmark, we apply traditional 3D reconstruction to register the photos and approximate the scene as a point cloud. For each photo, we render the scene points into a deep framebuffer, and train a neural network to learn the mapping of these initial renderings to the actual photos. This rerendering network also takes as input a latent appearance vector and a semantic mask indicating the location of transient objects like pedestrians. The model is evaluated on several datasets of publicly available images spanning a broad range of illumination conditions. We create short videos demonstrating realistic manipulation of the image viewpoint, appearance,and semantic labeling. We also compare results with prior work on scene reconstruction from internet photos.

引用

页码：3871 / 6880

页数：3010