A CNN-Based Online Self-Calibration of Binocular Stereo Cameras for Pose Change

This article proposes a novel method that can automatically and accurately recognize the pose change of binocular stereo cameras in real time and correct these changes. Focused on predicting a five degree-of-freedom extrinsic pose, we design a convolutional neural network (CNN) that implements the regression of rotation angles of two cameras. The proposed method increases regression accuracy using the information inherent in the entire image. To this end, the CNN divides the image into patches of a certain size, extracts detailed features and context features of the patches, and extracts attention information for patches belonging to the left and right images. Training and evaluating the CNN requires many stereo images with variations from the initial setup of the cameras. We solve this problem using miscalibration. In miscalibration, angles expected to be rotated for the three axes of the left and right cameras are randomly sampled within a range of +/- 2.5 degrees, and a pair of rectified images are transformed using the sampled angles. The CNN uses these transformed images to infer the angle at which the camera axis is expected to have been rotated. Then, the pair of transformed images are corrected with these inferred angles. The superiority of the proposed method is demonstrated using the KITTI odometry dataset and the GY dataset we built.