Secure content based image retrieval for mobile users with deep neural networks in the cloud

With the tremendous growth of smart mobile devices, the Content-Based Image Retrieval (CBIR) becomes popular and has great market potentials. Secure image retrieval has attracted considerable interests recently due to the outsourcing of CBIR onto the cloud. In this paper, we propose and implement a secure CBIR framework that performs image retrieval on the cloud without the user's interaction. A pre-trained generic DNN model (e.g., VGG-16) is used to extract the feature vectors of an image on the user side. The cloud servers perform secure image inference with a private pre-trained DNN model and execute Approximate Nearest Neighbor (ANN) image retrieval protocols without the user's anymore interaction. We design and implement a set of protocols for the secure evaluation of the non-linear functions in DNNs. The information about the image contents, the private DNN model parameters, the intermediate and the retrieval results is strictly concealed by the conjunctive use of the lattice-based homomorphic scheme and two-party computation (2-PC) techniques. We further propose a secure image similarity scoring and a result sorting protocols, which enable the cloud servers to compare and sort images without knowing any information about their features or contents. The comprehensive experimental results show that our framework is efficient, accurate and secure.