3D scene reconstruction using a texture probabilistic grammar

In this paper, texture probabilistic grammar is defined for the first time. We have developed an algorithm to obtain the 3D information in a 2D scene by training the texture probabilistic grammar from the prebuilt model library. The well-trained texture probabilistic grammar could also be applied to 3D reconstruction. Our detailed process contains: dividing the 2D scene into texture fragments; assigning the most suitable 3D object label to the 2D texture fragments; using our texture probabilistic grammar to predict 3D information of the texture fragments in 2D scene image; constructing the 3D model of the original 2D scene image. Through experiments, it is proved that the algorithm has a better effect on reconstruction of indoor scenes and building structures, and the algorithm is superior to the traditional reconstruction method based on point clouds. Different datasets and reconstructed objects are tested, which verifies the robustness of the algorithm. As a result, our algorithm is able to deal with the large numbers of scenes with similar semantics and it is also fast enough to deal with the online 3D reconstruction.