Multi-Scale Geometric Consistency Guided and Planar Prior Assisted Multi-View Stereo

被引:52
作者
Xu, Qingshan [1 ]
Kong, Weihang [1 ]
Tao, Wenbing [1 ]
Pollefeys, Marc [2 ,3 ]
机构
[1] Huazhong Univ Sci & Technol, Sch Artificial Intelligence & Automat, Natl Key Lab Sci & Technol Multispectral Informat, Wuhan, Peoples R China
[2] Swiss Fed Inst Technol, Dept Comp Sci, CH-8092 Zurich, Switzerland
[3] Microsoft, Redmond, WA 98052 USA
基金
中国国家自然科学基金;
关键词
Costs; Estimation; Three-dimensional displays; Parallel processing; Probabilistic logic; Pipelines; Solid modeling; Multi-view stereo; structured region information; multi-scale scheme; geometric consistency guidance; planar prior assistance; low-textured areas; 3D RECONSTRUCTION; GRAPH-CUTS; SILHOUETTE;
D O I
10.1109/TPAMI.2022.3200074
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we propose some efficient multi-view stereo methods for accurate and complete depth map estimation. We first present our basic methods with Adaptive Checkerboard sampling and Multi-Hypothesis joint view selection (ACMH $\&$& ACMH+). Based on our basic models, we develop two frameworks to deal with the depth estimation of ambiguous regions (especially low-textured areas) from two different perspectives: multi-scale information fusion and planar geometric clue assistance. For the former one, we propose a multi-scale geometric consistency guidance framework (ACMM) to obtain the reliable depth estimates for low-textured areas at coarser scales and guarantee that they can be propagated to finer scales. For the latter one, we propose a planar prior assisted framework (ACMP). We utilize a probabilistic graphical model to contribute a novel multi-view aggregated matching cost. At last, by taking advantage of the above frameworks, we further design a multi-scale geometric consistency guided and planar prior assisted multi-view stereo (ACMMP). This greatly enhances the discrimination of ambiguous regions and helps their depth sensing. Experiments on extensive datasets show our methods achieve state-of-the-art performance, recovering the depth estimation not only in low-textured areas but also in details. Related codes are available at https://github.com/GhiXu.
引用
收藏
页码:4945 / 4963
页数:19
相关论文
共 50 条
[41]   Phase-Based Window Matching with Geometric Correction for Multi-View Stereo [J].
Sakai, Shuji ;
Ito, Koichi ;
Aoki, Takafumi ;
Watanabe, Takafumi ;
Unten, Hiroki .
IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2015, E98D (10) :1818-1828
[42]   A TV Prior for High-Quality Scalable Multi-View Stereo Reconstruction [J].
Kuhn, Andreas ;
Hirschmueller, Heiko ;
Scharstein, Daniel ;
Mayer, Helmut .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 2017, 124 (01) :2-17
[43]   A TV Prior for High-Quality Scalable Multi-View Stereo Reconstruction [J].
Andreas Kuhn ;
Heiko Hirschmüller ;
Daniel Scharstein ;
Helmut Mayer .
International Journal of Computer Vision, 2017, 124 :2-17
[44]   Feature distribution normalization network for multi-view stereo [J].
Chen, Ziyang ;
Zhao, Yang ;
He, Junling ;
Lu, Yujie ;
Cui, Zhongwei ;
Li, Wenting ;
Zhang, Yongjun .
VISUAL COMPUTER, 2025, 41 (01) :409-421
[45]   Efficient large-scale multi-view stereo for ultra high-resolution image sets [J].
Tola, Engin ;
Strecha, Christoph ;
Fua, Pascal .
MACHINE VISION AND APPLICATIONS, 2012, 23 (05) :903-920
[46]   Multi-view photometric stereo using surface deformation [J].
Gan, Jiangbin ;
Wilbert, Alwin ;
Thormaehlen, Thorsten ;
Drescher, Philip ;
Hagens, Ralf .
VISUAL COMPUTER, 2018, 34 (11) :1551-1561
[47]   Adaptive depth estimation for pyramid multi-view stereo [J].
Liao, Jie ;
Fu, Yanping ;
Yan, Qingan ;
Luo, Fei ;
Xiao, Chunxia .
COMPUTERS & GRAPHICS-UK, 2021, 97 :268-278
[48]   An Effective Energy-based Multi-view Piecewise Planar Stereo Method [J].
Zhu, Hai ;
Wang, Wei ;
Wang, Xiaoli ;
Wei, Fei .
2015 11TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND SECURITY (CIS), 2015, :117-120
[49]   Semi-supervised Deep Multi-view Stereo [J].
Xu, Hongbin ;
Chen, Weitao ;
Liu, Yang ;
Zhou, Zhipeng ;
Xiao, Haihong ;
Sun, Baigui ;
Xie, Xuansong ;
Kang, Wenxiong .
PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, :4616-4625
[50]   Multi-View Stereo Network With Gaussian Distribution Iteration [J].
Zhang, Xiaohan ;
Li, Shikun .
IEEE ACCESS, 2023, 11 :53359-53372