Multi-view stereo of an object immersed in a refractive medium

被引:0
作者
Bruneau, Robin [1 ,2 ]
Brument, Baptiste [1 ]
Calvet, Lilian [3 ]
Cassidy, Matthew [1 ]
Melou, Jean [1 ]
Queau, Yvain [4 ]
Durou, Jean-Denis [1 ]
Lauze, Francois [2 ]
机构
[1] Univ Toulouse, UMR CNRS 5505, IRIT, Toulouse, France
[2] DIKU, Dept Comp Sci, Copenhagen, Denmark
[3] Univ Zurich, Balgrist Hosp, OR X, Zurich, Switzerland
[4] Normandie Univ, UNICAEN, CNRS, ENSICAEN,GREYC, Caen, France
关键词
three-dimensional reconstruction; multi-view stereo; refraction; LIGHT; DEPTH; POSE;
D O I
10.1117/1.JEI.33.3.033005
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
. In this article, we show how to extend the multi-view stereo technique when the object to be reconstructed is inside a transparent but refractive medium, which causes distortions in the images. We provide a theoretical formulation of the problem accounting for a general non-planar shape of the refractive interface, and then a discrete solving method. We also present a pipeline to recover precisely the geometry of the refractive interface, considered as a convex polyhedral object. It is based on the extraction of visible polyhedron vertices from silhouette images and matching across a sequence of images acquired under circular camera motion. These contributions are validated by tests on synthetic and real data.
引用
收藏
页数:22
相关论文
共 60 条
[31]  
Jordt-Sedlazeck A, 2013, LECT NOTES COMPUT SC, V8142, P333, DOI 10.1007/978-3-642-40602-7_36
[32]  
Jordt-Sedlazeck A, 2012, LECT NOTES COMPUT SC, V7576, P846, DOI 10.1007/978-3-642-33715-4_61
[33]  
Kang L, 2012, LECT NOTES COMPUT SC, V7575, P303, DOI 10.1007/978-3-642-33765-9_22
[34]  
Kirillov A, 2023, Arxiv, DOI arXiv:2304.02643
[35]   Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes [J].
Li, Zhengqin ;
Yeh, Yu-Ying ;
Chandraker, Manmohan .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :1259-1268
[36]   Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning [J].
Liu, Shichen ;
Li, Tianye ;
Chen, Weikai ;
Li, Hao .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :7707-7716
[37]   Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects [J].
Lyu, Jiahui ;
Wu, Bojian ;
Lischinski, Dani ;
Cohen-Or, Daniel ;
Huang, Hui .
ACM TRANSACTIONS ON GRAPHICS, 2020, 39 (06)
[38]  
Maas H. G., 1995, Optical 3-D Measurement Techniques, VIII
[39]   Dynamic Refraction Stereo [J].
Morris, Nigel J. W. ;
Kutulakos, Kiriakos N. .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2011, 33 (08) :1518-1531
[40]   Extracting Triangular 3D Models, Materials, and Lighting From Images [J].
Munkberg, Jacob ;
Chen, Wenzheng ;
Hasselgren, Jon ;
Evans, Alex ;
Shen, Tianchang ;
Muller, Thomas ;
Gao, Jun ;
Fidler, Sanja .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, :8270-8280