3D Street Object Detection from Monocular Images Using Deep Learning and Depth Information

In this study, we present a three-dimensional (3D) ob-ject detection algorithm based on monocular images by constructing an end-to-end network, that incorpo-rates depth information. The entire network consists of three parts. The first part includes the basic ob-ject detection neural network as the main body, that uses the region proposal network to obtain the two-dimensional (2D) region proposal of the object. The second part is the depth estimation branch network, that obtains the depth information of the object pix-els and calculates the corresponding 3D point cloud. In the last part, concatenated features obtained from the aforementioned two parts are fed into the fully -connected layers. Subsequently, 2D and 3D detection results are obtained. Compared with certain existing methods, the accuracy of the detection results is im-proved in this study.