Integrated Deconvolution Keypoint Detector and Descriptor Network

In this paper, we introduce a deconvolution-based approach to learn detector and descriptor for repeatable keypoints under drastic environmental changes of weather and lighting conditions. As opposed to patch-based neural network, our integrated deconvolution keypoint detector and descriptor (IDKDD) network operates on full-sized images and jointly computes pixel-level keypoint locations and associated descriptors in one forward pass at both training and testing stages. We introduce the deconvolution-based detector and descriptor considering both low-level and high-level information of CNN network to deal with the challenging issue, temporal variations. We conduct comprehensive quantitative and qualitative experiments on standard dataset and demonstrate that our IDKDD network significantly outperforms the state-of-the-art methods on keypoint detection and description.