gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction

被引:17
作者
Chen, Zerui [1 ]
Chen, Shizhe [1 ]
Schmid, Cordelia [1 ]
Laptev, Ivan [1 ]
机构
[1] PSL Res Univ, CNRS, INRIA, Ecole Normale Super, F-75005 Paris, France
来源
2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) | 2023年
关键词
D O I
10.1109/CVPR52729.2023.01239
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Signed distance functions (SDFs) is an attractive framework that has recently shown promising results for 3D shape reconstruction from images. SDFs seamlessly generalize to different shape resolutions and topologies but lack explicit modelling of the underlying 3D geometry. In this work, we exploit the hand structure and use it as guidance for SDF-based shape reconstruction. In particular, we address reconstruction of hands and manipulated objects from monocular RGB images. To this end, we estimate poses of hands and objects and use them to guide 3D reconstruction. More specifically, we predict kinematic chains of pose transformations and align SDFs with highly-articulated hand poses. We improve the visual features of 3D points with geometry alignment and further leverage temporal information to enhance the robustness to occlusion and motion blurs. We conduct extensive experiments on the challenging ObMan and DexYCB benchmarks and demonstrate significant improvements of the proposed method over the state of the art.
引用
收藏
页码:12890 / 12900
页数:11
相关论文
共 67 条
[1]  
[Anonymous], 2009, ICCV
[2]  
[Anonymous], 2019, CVPR, DOI DOI 10.1109/CVPR.2019.00464
[3]  
[Anonymous], 2017, CVPR, DOI DOI 10.1109/CVPR.2017.138
[4]  
[Anonymous], 2011, ICCV
[5]  
[Anonymous], 2018, CVPR, DOI DOI 10.1109/CVPR.2018.00279
[6]  
[Anonymous], 2019, CVPR, DOI DOI 10.1109/CVPR.2019.01110
[7]  
[Anonymous], 2019, CVPR, DOI DOI 10.1109/CVPR.2019.00352
[8]  
[Anonymous], 2017, TOG, DOI DOI 10.1145/3130800.3130883
[9]  
Arnab Anurag, 2021, ICCV
[10]   Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Rendering [J].
Baek, Seungryul ;
Kim, Kwang In ;
Kim, Tae-Kyun .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :1067-1076