NeRF-based Polarimetric Multi-view Stereo

In this paper, we introduce NeRF-based Polarimetric Multi-view Stereo (NPMVS), a novel 3D reconstruction method that combines the advantages of neural radiance field (NeRF) and shape-from-polarization (SfP) address the challenge posed by textureless areas while preserving the fine-scale geometric details. Our method first leverages neural rendering to yield depth priors for each input view, subsequently estimates more accurate depths and normals using polarimetric refinement. We further introduce a pixel-wise depth rectification process to address the scaling problem inherent to the polarimetric refinement procedure. In addition, we contribute new realistic pBRDF-based multi-view synthetic dataset, comprised of RGB and polarization images rendered under real-world lighting conditions, which will serve as a valuable resource for future research in this field. Experimental evaluations on both synthetic and real-world datasets validate the superiority of NPMVS, demonstrating its advantage over other state-of-the-art multi-view stereo and shape-from-polarization methods.