FEW SHOT PHOTOGRAMETRY: A COMPARISON BETWEEN NERF AND MVS-SFM FOR THE DOCUMENTATION OF CULTURAL HERITAGE

被引:12
作者
Balloni, E. [1 ]
Gorgoglione, L. [2 ]
Paolanti, M. [3 ]
Mancini, A. [1 ]
Pierdicca, R. [2 ]
机构
[1] Univ Politecn Marche, VRAI Lab, Dipartimento Ingn dellInformaz DII, I-60131 Ancona, Italy
[2] Univ Politecn Marche, Dipartimento Ingn Civile, Edile dellArchitettura, I-60131 Ancona, Italy
[3] Univ Macerata, Dept Polit Sci Commun & Int Relat, I-62100 Macerata, Italy
来源
29TH CIPA SYMPOSIUM DOCUMENTING, UNDERSTANDING, PRESERVING CULTURAL HERITAGE. HUMANITIES AND DIGITAL TECHNOLOGIES FOR SHAPING THE FUTURE, VOL. 48-M-2 | 2023年
关键词
NeRF; Deep Learning; Photogrammetry; Documentation; Cultural Heritage; Artificial Intelligence; 3D Reconstruction;
D O I
10.5194/isprs-archives-XLVIII-M-2-2023-155-2023
中图分类号
K85 [文物考古];
学科分类号
0601 ;
摘要
3D documentation methods for Digital Cultural Heritage (DCH) domain is a field that becomes increasingly interdisciplinary, breaking down boundaries that have long separated experts from different domains. In the past, there has been an ambiguous claim for ownership of skills, methodologies, and expertise in the heritage sciences. This study aims to contribute to the dialogue between these different disciplines by presenting a novel approach for 3D documentation of an ancient statue. The method combines TLS acquisition and MVS pipeline using images from a DJI Mavic 2 drone. Additionally, the study compares the accuracy and final product of the Deep Points (DP) and Neural Radiance Fields (NeRF) methods, using the TLS acquisition as validation ground truth. Firstly, a TLS acquisition was performed on an ancient statue using a Faro Focus 2 scanner. Next, a multi-view stereo (MVS) pipeline was adopted using 2D images captured by a Mini-2 DJI Mavic 2 drone from a distance of approximately 1 meter around the statue. Finally, the same images were used to train and run the NeRF network after being reduced by 90%. The main contribution of this paper is to improve our understanding of this method and compare the accuracy and final product of two different approaches - direct projection (DP) and NeRF - by exploiting a TLS acquisition as the validation ground truth. Results show that the NeRF approach outperforms DP in terms of accuracy and produces a more realistic final product. This paper has important implications for the field of CH preservation, as it offers a new and effective method for generating 3D models of ancient statues. This technology can help to document and preserve important cultural artifacts for future generations, while also providing new insights into the history and culture of different civilizations. Overall, the results of this study demonstrate the potential of combining TLS and NeRF for generating accurate and realistic 3D models of ancient statues.
引用
收藏
页码:155 / 162
页数:8
相关论文
共 36 条
[1]   Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields [J].
Barron, Jonathan T. ;
Mildenhall, Ben ;
Tancik, Matthew ;
Hedman, Peter ;
Martin-Brualla, Ricardo ;
Srinivasan, Pratul P. .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :5835-5844
[2]  
Bebis G, 2011, 7 INT S ISVC 2011 LA
[3]  
Condorelli F., 2021, Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci, V43, P565, DOI [10.5194/isprs-archives-XLIII-B2-2021-565-2021, DOI 10.5194/ISPRS-ARCHIVES-XLIII-B2-2021-565-2021]
[4]  
Deng B., 2020, JaxNeRF: an efficient JAX implementation of NeRF
[5]   Knowledge graph based on domain ontology and natural language processing technology for Chinese intangible cultural heritage [J].
Dou, Jinhua ;
Qin, Jingyan ;
Jin, Zanxia ;
Li, Zhuang .
JOURNAL OF VISUAL LANGUAGES AND COMPUTING, 2018, 48 :19-28
[6]   MO.SE.: MOSAIC IMAGE SEGMENTATION BASED ON DEEP CASCADING LEARNING [J].
Felicetti, Andrea ;
Paolanti, Marina ;
Zingaretti, Primo ;
Pierdicca, Roberto ;
Malinverni, Eva Savina .
VIRTUAL ARCHAEOLOGY REVIEW, 2021, 12 (24) :25-38
[7]   Machine Learning for Cultural Heritage: A Survey [J].
Fiorucci, Marco ;
Khoroshiltseva, Marina ;
Pontil, Massimiliano ;
Traviglia, Arianna ;
Del Bue, Alessio ;
James, Stuart .
PATTERN RECOGNITION LETTERS, 2020, 133 :102-108
[8]  
Gao K., 2022, Nerf: Neural radiance field in 3d vision, a comprehensive review
[9]  
Garbin S. J., 2021, Fastnerf: High-fidelity neural rendering at 200fps
[10]   Machine Learning Generalisation across Different 3D Architectural Heritage [J].
Grilli, Eleonora ;
Remondino, Fabio .
ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, 2020, 9 (06)