Adaptive Feature Enhanced Multi-View Stereo With Epipolar Line Information Aggregation

被引:2
作者
Wang, Shaoqian [1 ,2 ]
Li, Bo [1 ,2 ]
Yang, Jian [3 ]
Dai, Yuchao [1 ,2 ]
机构
[1] Northwestern Polytech Univ, Sch Elect & Informat, Xian 710129, Peoples R China
[2] Northwestern Polytech Univ, Shaanxi Key Lab Informat Acquisit & Proc, Xian 710129, Peoples R China
[3] Rocket Force Univ Engn, Xian 710025, Peoples R China
基金
中国国家自然科学基金;
关键词
Correlation; Feature extraction; Costs; Three-dimensional displays; Image reconstruction; Aggregates; Robustness; Data mining; Visualization; Estimation; Epipolar line information aggregation (EIA); feature enhancement; multi-view stereo; perspective transformation;
D O I
10.1109/LRA.2024.3471454
中图分类号
TP24 [机器人技术];
学科分类号
080202 ; 1405 ;
摘要
Despite the promising performance achieved by the learning-based multi-view stereo (MVS) methods, the commonly used feature extractors still struggle with the perspective transformation across different viewpoints. Furthermore, existing methods generally employ a "one-to-many" strategy, computing the correlations between the fixed reference image feature and multiple source image features, which limits the diversity of feature enhancement for the reference image. To address these issues, we propose a novel Epipolar Line Information Aggregati(EIA) method. Specifically, we present a feature enhancement layer (EIA-F) that utilizes the epipolar line information to enhance image features. EIA-F employs a "many-to-many" strategy, adaptively enhancing the reference-source feature pairs with diverse epipolar line information. Additionally, we propose a correlation enhancement module (EIA-C) to improve the robustness of correlations. Extensive experiments demonstrate that our method achieves state-of-the-art performance across multiple MVS benchmarks, particularly in terms of reconstruction integrity.
引用
收藏
页码:10439 / 10446
页数:8
相关论文
共 36 条
[11]  
Kingma Diederik P, 2014, ARXIV PREPRINT ARXIV
[12]   Tanks and Temples: Benchmarking Large-Scale Scene Reconstruction [J].
Knapitsch, Arno ;
Park, Jaesik ;
Zhou, Qian-Yi ;
Koltun, Vladlen .
ACM TRANSACTIONS ON GRAPHICS, 2017, 36 (04)
[13]   Feature Pyramid Networks for Object Detection [J].
Lin, Tsung-Yi ;
Dollar, Piotr ;
Girshick, Ross ;
He, Kaiming ;
Hariharan, Bharath ;
Belongie, Serge .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :936-944
[14]   When Epipolar Constraint Meets Non-local Operators in Multi-View Stereo [J].
Liu, Tianqi ;
Ye, Xinyi ;
Zhao, Weiyue ;
Pan, Zhiyu ;
Shi, Min ;
Cao, Zhiguo .
2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, :18042-18051
[15]   P-MVSNet: Learning Patch-wise Matching Confidence Aggregation for Multi-View Stereo [J].
Luo, Keyang ;
Guan, Tao ;
Ju, Lili ;
Huang, Haipeng ;
Luo, Yawei .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :10451-10460
[16]   EPP-MVSNet: Epipolar-assembling based Depth Prediction for Multi-view Stereo [J].
Ma, Xinjun ;
Gong, Yue ;
Wang, Qirui ;
Huang, Jingwei ;
Chen, Lei ;
Yu, Fan .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :5712-5720
[17]   DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volume [J].
Miao, Xingyu ;
Bai, Yang ;
Duan, Haoran ;
Huang, Yawen ;
Wan, Fan ;
Xu, Xinxing ;
Long, Yang ;
Zheng, Yefeng .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2024, 34 (04) :2564-2576
[18]   Don't Forget The Past: Recurrent Depth Estimation from Monocular Video [J].
Patil, Vaishakh ;
Van Gansbeke, Wouter ;
Dai, Dengxin ;
Van Gool, Luc .
IEEE ROBOTICS AND AUTOMATION LETTERS, 2020, 5 (04) :6813-6820
[19]   View Planning for Multi-View Stereo 3D Reconstruction Using an Autonomous Multicopter [J].
Schmid, Korbinian ;
Hirschmueller, Heiko ;
Doemel, Andreas ;
Grixa, Iris ;
Suppa, Michael ;
Hirzinger, Gerd .
JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS, 2012, 65 (1-4) :309-323
[20]   Structure-from-Motion Revisited [J].
Schonberger, Johannes L. ;
Frahm, Jan -Michael .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :4104-4113