MVSGS: Gaussian splatting radiation field enhancement using multi-view stereo

被引:0
作者
Fei, Teng [1 ]
Bi, Ligong [2 ]
Gao, Jieming [3 ]
Chen, Shuixuan [1 ]
Zhang, Guowei [1 ]
机构
[1] Xiamen Univ Technol, Sch Mech & Automot Engn, 600 Ligong Rd, Xiamen 361024, Fujian, Peoples R China
[2] Guilin Univ Elect Technol, Guilin 541000, Guangxi, Peoples R China
[3] Tianjin Univ, Tianjin 300072, Peoples R China
关键词
3D reconstruction; 3D Gaussian splatting; Multi-view stereo; Radiation field; Explicit geometry; Density optimization; Avatar reconstruction; 3D GAUSSIANS; AVATAR;
D O I
10.1007/s40747-024-01691-x
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the advent of 3D Gaussian Splatting (3DGS), new and effective solutions have emerged for 3D reconstruction pipelines and scene representation. However, achieving high-fidelity reconstruction of complex scenes and capturing low-frequency features remain long-standing challenges in the field of visual 3D reconstruction. Relying solely on sparse point inputs and simple optimization criteria often leads to non-robust reconstructions of the radiance field, with reconstruction quality heavily dependent on the proper initialization of inputs. Notably, Multi-View Stereo (MVS) techniques offer a mature and reliable approach for generating structured point cloud data using a limited number of views, camera parameters, and feature matching. In this paper, we propose combining MVS with Gaussian Splatting, along with our newly introduced density optimization strategy, to address these challenges. This approach bridges the gap in scene representation by enhancing explicit geometry radiance fields with MVS, and our experimental results demonstrate its effectiveness. Additionally, we have explored the potential of using Gaussian Splatting for non-face template single-process end-to-end Avatar Reconstruction, yielding promising experimental results.
引用
收藏
页数:17
相关论文
共 55 条
[1]   PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing [J].
Barnes, Connelly ;
Shechtman, Eli ;
Finkelstein, Adam ;
Goldman, Dan B. .
ACM TRANSACTIONS ON GRAPHICS, 2009, 28 (03)
[2]   Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields [J].
Barron, Jonathan T. ;
Mildenhall, Ben ;
Verbin, Dor ;
Srinivasan, Pratul P. ;
Hedman, Peter .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :5460-5469
[3]   PatchMatch Stereo - Stereo Matching with Slanted Support Windows [J].
Bleyer, Michael ;
Rhemann, Christoph ;
Rother, Carsten .
PROCEEDINGS OF THE BRITISH MACHINE VISION CONFERENCE 2011, 2011,
[4]   From 3D reconstruction to virtual reality: A complete methodology for digital archaeological exhibition [J].
Bruno, Fabio ;
Bruno, Stefano ;
De Sensi, Giovanna ;
Luchi, Maria-Laura ;
Mancuso, Stefania ;
Muzzupappa, Maurizio .
JOURNAL OF CULTURAL HERITAGE, 2010, 11 (01) :42-49
[5]   pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction [J].
Charatan, David ;
Li, Sizhe Lester ;
Tagliasacchi, Andrea ;
Sitzmann, Vincent .
2024 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2024, :19457-19467
[6]   MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [J].
Chen, Anpei ;
Xu, Zexiang ;
Zhao, Fuqiang ;
Zhang, Xiaoshuai ;
Xiang, Fanbo ;
Yu, Jingyi ;
Su, Hao .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :14104-14113
[7]   Point-Based Multi-View Stereo Network [J].
Chen, Rui ;
Han, Songfang ;
Xu, Jing ;
Su, Hao .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :1538-1547
[8]   Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness [J].
Cheng, Shuo ;
Xu, Zexiang ;
Zhu, Shilin ;
Li, Zhuwen ;
Li, Li Erran ;
Ramamoorthi, Ravi ;
Su, Hao .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :2521-2531
[9]   Approximate development of trimmed patches for surface tessellation [J].
Cho, WJ ;
Patrikalakis, NM ;
Peraire, J .
COMPUTER-AIDED DESIGN, 1998, 30 (14) :1077-1087
[10]   Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis [J].
Dai, Angela ;
Qi, Charles Ruizhongtai ;
Niessner, Matthias .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :6545-6554