GS-SFS: Joint Gaussian Splatting and Shape-From-Silhouette for Multiple Human Reconstruction in Large-Scale Sports Scenes

被引:0
|
作者
Jiang, Yuqi [1 ]
Li, Jing [1 ]
Qin, Haidong [2 ]
Dai, Yanran [1 ]
Liu, Jing [1 ]
Zhang, Guodong [1 ]
Zhang, Canbin [1 ]
Yang, Tao [2 ]
机构
[1] Xidian Univ, Sch Telecommun Engn, Xidian 710071, Peoples R China
[2] Northwestern Polytech Univ, Sch Comp Sci, Natl Engn Lab Integrated Aerosp Ground Ocean Big D, SAIIP, Xian 710129, Peoples R China
基金
中国国家自然科学基金;
关键词
Image reconstruction; Three-dimensional displays; Sports; Surface reconstruction; Cameras; Accuracy; Solid modeling; 3D Gaussian splatting; human body reconstruction; multi-view reconstruction; shape-from-silhouette; virtual view synthesis; FREE-VIEWPOINT VIDEO; GENERATION; TIME;
D O I
10.1109/TMM.2024.3443637
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
We introduce GS-SFS, a method that utilizes a camera array with wide baselines for high-quality multiple human mesh reconstruction in large-scale sports scenes. Traditional human reconstruction methods in sports scenes, such as Shape-from-Silhouette (SFS), struggle with sparse camera setups and small human targets, making it challenging to obtain complete and accurate human representations. Despite advances in differentiable rendering, including 3D Gaussian Splatting (3DGS), which can produce photorealistic novel-view renderings with dense inputs, accurate depiction of surfaces and generation of detailed meshes is still challenging. Our approach uniquely combines 3DGS's view synthesis with an optimized SFS method, thereby significantly enhancing the quality of multiperson mesh reconstruction in large-scale sports scenes. Specifically, we introduce body shape priors, including the human surface point clouds extracted through SFS and human silhouettes, to constrain 3DGS to a more accurate representation of the human body only. Then, we develop an improved mesh reconstruction method based on SFS, mainly by adding additional viewpoints through 3DGS and obtaining a more accurate surface to achieve higher-quality reconstruction models. We implement a high-density scene resampling strategy based on spherical sampling of human bounding boxes and render new perspectives using 3D Gaussian Splatting to create precise and dense multi-view human silhouettes. During mesh reconstruction, we integrate the human body's 2D Signed Distance Function (SDF) into the computation of the SFS's implicit surface field, resulting in smoother and more accurate surfaces. Moreover, we enhance mesh texture mapping by blending original and rendered images with different weights, preserving high-quality textures while compensating for missing details. The experimental results from real basketball game scenarios demonstrate the significant improvements of our approach for multiple human body model reconstruction in complex sports settings.
引用
收藏
页码:11095 / 11110
页数:16
相关论文
共 2 条
  • [1] LI-GS: Gaussian Splatting With LiDAR Incorporated for Accurate Large-Scale Reconstruction
    Jiang, Changjian
    Gao, Ruilan
    Shao, Kele
    Wang, Yue
    Xiong, Rong
    Zhang, Yu
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2025, 10 (02): : 1864 - 1871
  • [2] Unbounded-GS: Extending 3D Gaussian Splatting With Hybrid Representation for Unbounded Large-Scale Scene Reconstruction
    Li, Wanzhang
    Yin, Fukun
    Liu, Wen
    Yang, Yiying
    Chen, Xin
    Jiang, Biao
    Yu, Gang
    Fan, Jiayuan
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2024, 9 (12): : 11529 - 11536