Drone-NeRF: Efficient NeRF based 3D scene reconstruction for large-scale drone survey

被引:12
作者
Jia, Zhihao [1 ]
Wang, Bing [2 ]
Chen, Changhao [3 ]
机构
[1] Univ Southampton, Geog & Environm, Southampton SO17 1BJ, Hants, England
[2] Hong Kong Polytech Univ, Dept Aeronaut & Aviat Engn, Hong Kong, Peoples R China
[3] Natl Univ Def Technol, Coll Intelligence Sci & Technol, Changsha, Peoples R China
基金
中国国家自然科学基金;
关键词
Scene reconstruction; Neural radiance fields; UAV; STEREO; SHAPE;
D O I
10.1016/j.imavis.2024.104920
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Neural rendering has garnered substantial attention owing to its capacity for creating realistic 3D scenes. However, its applicability to extensive scenes remains challenging, with limitations in effectiveness. In this work, we propose the Drone-NeRF framework to enhance the efficient reconstruction of unbounded large-scale scenes suited for drone oblique photography using Neural Radiance Fields (NeRF). Our approach involves dividing the scene into uniform sub-blocks based on camera position and depth visibility. Sub-scenes are trained in parallel using NeRF, then merged for a complete scene. We refine the model by optimizing camera poses and guiding NeRF with a uniform sampler. Integrating chosen samples enhances accuracy. A hash-coded fusion MLP accelerates density representation, yielding RGB and Depth outputs. Our framework accounts for sub-scene constraints, reduces parallel-training noise, handles shadow occlusion, and merges sub-regions for a polished rendering result. Moreover, our framework can be enhanced through the integration of semantic scene division, ensuring consistent allocation of identical objects to the same sub-block for improved object integrity and rendering performance. This Drone-NeRF framework demonstrates promising capabilities in addressing challenges related to scene complexity, rendering efficiency, and accuracy in drone-obtained imagery.
引用
收藏
页数:17
相关论文
共 49 条
[1]   RigNeRF: Fully Controllable Neural 3D Portraits [J].
Athar, ShahRukh ;
Xu, Zexiang ;
Sunkavalli, Kalyan ;
Shechtman, Eli ;
Shu, Zhixin .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :20332-20341
[2]  
Ba J, 2014, ACS SYM SER
[3]   Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields [J].
Barron, Jonathan T. ;
Mildenhall, Ben ;
Verbin, Dor ;
Srinivasan, Pratul P. ;
Hedman, Peter .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :5460-5469
[4]   Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields [J].
Barron, Jonathan T. ;
Mildenhall, Ben ;
Tancik, Matthew ;
Hedman, Peter ;
Martin-Brualla, Ricardo ;
Srinivasan, Pratul P. .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :5835-5844
[5]   Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation [J].
Cai, Shengqu ;
Obukhov, Anton ;
Dai, Dengxin ;
Van Gool, Luc .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :3971-3980
[6]  
Carr JC, 2001, COMP GRAPH, P67, DOI 10.1145/383259.383266
[7]   Review of key technologies for warehouse 3D reconstruction [J].
Hao, Y. N. ;
Tan, Y. C. ;
Tai, V. C. ;
Zhang, X. D. ;
Wei, E. P. ;
Ng, S. C. .
JOURNAL OF MECHANICAL ENGINEERING AND SCIENCES, 2022, 16 (03) :9142-9156
[8]   High Accuracy and Visibility-Consistent Dense Multiview Stereo [J].
Hoang-Hiep Vu ;
Labatut, Patrick ;
Pons, Jean-Philippe ;
Keriven, Renaud .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2012, 34 (05) :889-901
[9]   THE VARIATIONAL APPROACH TO SHAPE FROM SHADING [J].
HORN, BKP ;
BROOKS, MJ .
COMPUTER VISION GRAPHICS AND IMAGE PROCESSING, 1986, 33 (02) :174-208
[10]   HDR-NeRF: High Dynamic Range Neural Radiance Fields [J].
Huang, Xin ;
Zhang, Qi ;
Feng, Ying ;
Li, Hongdong ;
Wang, Xuan ;
Wang, Qing .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, :18377-18387