3D depth computation of moving objects using a CNN with an orthogonal stereo fisheye camera system

Stereo matching of images taken by wide-angle (WA) fisheye lens cameras pose difficulties because the degree of distortion increases along the outer edge of the image. In addition, in an orthogonal stereo camera system in which adjacent WA cameras are positioned at right angles to cover 360 , the relationship between correspondence and depth information varies from that of a general-purpose parallel stereo camera system, thus requiring special mathematical modeling. In order to solve the abovementioned problems, this paper proposes an algorithm to minimize the degree of distortion by viewpoint transformation, and calculates the three-dimensional distance information of a moving object through the mathematical modeling of a fisheye stereo camera system arranged at right angles. The proposed algorithm consists of five steps. First, using the calibrated camera parameters, the viewpoints are changed so that the stereo images perpendicular to each other are viewed in parallel in the same direction. Second, by using a convolutional neural network (CNN) model, features are extracted from the viewpoint-transformed images. Third, matching between adjacent images based on the extracted features is performed. Fourth, depth information of moving objects is calculated from matching points. Finally, the computed depth information is refined for improved accuracy. Simulation results show that the depth calculated by the proposed algorithm is fairly accurate, with an average error rate of about 4%. ©2019 The Institute of Electronics and Information Engineers.