Single image 3D object reconstruction based on deep learning: A review

被引:78
作者
Fu, Kui [1 ]
Peng, Jiansheng [1 ,2 ]
He, Qiwen [1 ]
Zhang, Hanxiao [2 ]
机构
[1] Hechi Univ, Sch Phys & Mech & Elect Engn, Yizhou 546300, Guangxi, Peoples R China
[2] Guangxi Univ Sci & Technol, Sch Elect & Informat Engn, Liuzhou 545006, Guangxi, Peoples R China
关键词
Single image 3D reconstruction; Deep learning; Computer vision; 3D shape representation; SHAPE RECONSTRUCTION; FACE RECONSTRUCTION; STEREO;
D O I
10.1007/s11042-020-09722-8
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The reconstruction of 3D object from a single image is an important task in the field of computer vision. In recent years, 3D reconstruction of single image using deep learning technology has achieved remarkable results. Traditional methods to reconstruct 3D object from a single image require prior knowledge and assumptions, and the reconstruction object is limited to a certain category or it is difficult to accomplish a good reconstruction from a real image. Although deep learning can solve these problems well with its own powerful learning ability, it also faces many problems. In this paper, we first discuss the challenges faced by applying the deep learning method to reconstruct 3D objects from a single image. Second, we comprehensively review encoders, decoders and training details used in 3D reconstruction of a single image. Then, the common datasets and evaluation metrics of single image 3D object reconstruction in recent years are introduced. In order to analyze the advantages and disadvantages of different 3D reconstruction methods, a series of experiments are used for comparison. In addition, we simply give some related application examples involving 3D reconstruction of a single image. Finally, we summarize this paper and discuss the future directions.
引用
收藏
页码:463 / 498
页数:36
相关论文
共 151 条
[1]   Learning to Reconstruct People in Clothing from a Single RGB Camera [J].
Alldieck, Thiemo ;
Magnor, Marcus ;
Bhatnagar, Bharat Lal ;
Theobalt, Christian ;
Pons-Moll, Gerard .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :1175-1186
[2]  
[Anonymous], 2018, ACM T GRAPH SIGGRAPH
[3]  
[Anonymous], 2015, PROC CVPR IEEE, DOI DOI 10.1109/CVPR.2015.7298801
[4]  
[Anonymous], 2005, P BRIT MACH VIS C
[5]   Statistical approach to shape from shading: Reconstruction of three-dimensional face surfaces from single two-dimensional images [J].
Atick, JJ ;
Griffin, PA ;
Redlich, AN .
NEURAL COMPUTATION, 1996, 8 (06) :1321-1340
[6]   SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation [J].
Badrinarayanan, Vijay ;
Kendall, Alex ;
Cipolla, Roberto .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2017, 39 (12) :2481-2495
[7]   2D-3D shape reconstruction of the distal femur from stereo X-ray imaging using statistical shape models [J].
Baka, N. ;
Kaptein, B. L. ;
de Bruijne, M. ;
van Walsum, T. ;
Giphart, J. E. ;
Niessen, W. J. ;
Lelieveldt, B. P. F. .
MEDICAL IMAGE ANALYSIS, 2011, 15 (06) :840-850
[8]   Hierarchical Surface Prediction for 3D Object Reconstruction [J].
Bane, Christian ;
Tulsiani, Shubham ;
Malik, Jitendra .
PROCEEDINGS 2017 INTERNATIONAL CONFERENCE ON 3D VISION (3DV), 2017, :412-420
[9]   A morphable model for the synthesis of 3D faces [J].
Blanz, V ;
Vetter, T .
SIGGRAPH 99 CONFERENCE PROCEEDINGS, 1999, :187-194
[10]   Geometric Deep Learning Going beyond Euclidean data [J].
Bronstein, Michael M. ;
Bruna, Joan ;
LeCun, Yann ;
Szlam, Arthur ;
Vandergheynst, Pierre .
IEEE SIGNAL PROCESSING MAGAZINE, 2017, 34 (04) :18-42