Ground-to-Aerial Image Geo-Localization with Cross-View Image Synthesis

The task of ground-to-aerial image geo-localization can be achieved by matching a ground view query image to aerial images with geographic labels in a reference database. It remains challenging due to the drastic change in viewpoint. In this paper, we propose a new cross-view image synthesis conditional generative adversarial networks (cGAN) called Crossview Sequential Fork (CSF) to generate ground images from aerial images. CSF achieves a more detailed synthesis effect by the generation of segmentation maps and edge detection images. And the synthesis ground images are input to the image matching framework Cross View Synthesis Net (CVS-Net) to assist geo-localization, the distance between the descriptors of source ground image and synthesis ground image is calculated to assist the training of the network. CVS-Net is leveraged on the Siamese architecture to do metric learning for the matching task. Moreover, we introduce SARE loss as part of the training procedure and improve it by our data entry form which greatly improves the convergence rate and image retrieval accuracy compared to traditional triplet loss. Experimental results demonstrate the effectiveness and superiority of our proposed method over the state-of-the-art method on two benchmark datasets.