Geometry-Aware Single-Image Full-Body Human Relighting

被引:14
|
作者
Ji, Chaonan [1 ]
Yu, Tao [1 ]
Guo, Kaiwen [2 ]
Liu, Jingxin [3 ]
Liu, Yebin [1 ]
机构
[1] Tsinghua Univ, Dept Automat, Beijing, Peoples R China
[2] Meta Real Labs, Redmond, WA USA
[3] Guangdong OPPO Mobile Telecommun Corp Ltd, Dongguan, Peoples R China
来源
COMPUTER VISION - ECCV 2022, PT XVI | 2022年 / 13676卷
基金
国家重点研发计划;
关键词
Single-image human relighting; Single-image geometry reconstruction; Ray tracing; Neural rendering; LIGHT-SOURCE; SHAPE;
D O I
10.1007/978-3-031-19787-1_22
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Single-image human relighting aims to relight a target human under new lighting conditions by decomposing the input image into albedo, shape and lighting. Although plausible relighting results can be achieved, previous methods suffer from both the entanglement between albedo and lighting and the lack of hard shadows, which significantly decrease the realism. To tackle these two problems, we propose a geometry-aware single-image human relighting framework that leverages single-image geometry reconstruction for joint deployment of traditional graphics rendering and neural rendering techniques. For the de-lighting, we explore the shortcomings of UNet architecture and propose a modified HRNet, achieving better disentanglement between albedo and lighting. For the relighting, we introduce a ray tracing-based per-pixel lighting representation that explicitly models high-frequency shadows and propose a learning-based shading refinement module to restore realistic shadows (including hard cast shadows) from the ray-traced shading maps. Our framework is able to generate photo-realistic high-frequency shadows such as cast shadows under challenging lighting conditions. Extensive experiments demonstrate that our proposed method outperforms previous methods on both synthetic and real images.
引用
收藏
页码:388 / 405
页数:18
相关论文
共 5 条
  • [1] Relighting Humans: Occlusion-Aware Inverse Rendering for Full-Body Human Images
    Kanamori, Yoshihiro
    Endo, Yuki
    SIGGRAPH ASIA'18: SIGGRAPH ASIA 2018 TECHNICAL PAPERS, 2018,
  • [2] Relighting Humans: Occlusion-Aware Inverse Rendering for Full-Body Human Images
    Kanamori, Yoshihiro
    Endo, Yuki
    ACM TRANSACTIONS ON GRAPHICS, 2018, 37 (06):
  • [3] Tex2Shape: Detailed Full Human Body Geometry From a Single Image
    Alldieck, Thiemo
    Pons-Moll, Gerard
    Theobalt, Christian
    Magnor, Marcus
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 2293 - 2303
  • [4] Layout-aware Single-image Document Flattening
    Li, Pu
    Quan, Weize
    Guo, Jianwei
    Yan, Dong-Ming
    ACM TRANSACTIONS ON GRAPHICS, 2024, 43 (01):
  • [5] 3D Human Body Reconstruction from a Single Image via Volumetric Regression
    Jackson, Aaron S.
    Manafas, Chris
    Tzimiropoulos, Georgios
    COMPUTER VISION - ECCV 2018 WORKSHOPS, PT IV, 2019, 11132 : 64 - 77