DAIR: A Query-Efficient Decision-based Attack on Image Retrieval Systems

There is an increasing interest in studying adversarial attacks on image retrieval systems. However, most of the existing attack methods are based on the white-box setting, where the attackers have access to all the model and database details, which is a strong assumption for practical attacks. The generic transfer-based attack also requires substantial resources yet the effect was shown to be unreliable. In this paper, we make the first attempt in proposing a query-efficient decision-based attack framework for the image retrieval (DAIR) to completely subvert the top-Kappa retrieval results with human-imperceptible perturbations. We propose an optimization-based method with a smoothed utility function to overcome the challenging discrete nature of the problem. To further improve the query efficiency, we propose a novel sampling method that can achieve the transferability between the surrogate and the target model efficiently. Our comprehensive experimental evaluation on the benchmark datasets shows that our DAIR method outperforms significantly the state-of-the-art decision-based methods. We also demonstrate that real image retrieval engines (Bing Visual Search and Face++ engines) can be attacked successfully with only several hundreds of queries.