Efficient Secure CNN Inference: A Multi-Server Framework Based on Conditional Separable and Homomorphic Encryption

Deep learning inference has become a fundamental component of cloud service providers, while privacy issues during services have received significant attention. Although many privacy-preserving schemes have been proposed, they require further improvement. In this article, we propose Serpens, an efficient convolutional neural network (CNN) secure inference framework to protect users' uploaded data. We introduce a pair of novel concepts, namely separable and conditional separable, to determine whether a layer in CNNs can be computed over multiple servers or not. We demonstrate that linear layers are separable and construct factor-functions to reduce their overhead to nearly zero. For the two nonlinear layers, i.e., ReLU and max pooling, we design four secure protocols based on homomorphic encryption and random masks for two- and n-server settings. These protocols are essentially different from existing schemes, which are primarily based on garbled circuits. In addition, we extensively propose a method to split the image securely. The experimental results demonstrate that Serpens is 60x-197x faster than the previous scheme in the two-server setting. The superiority of Serpens is even more significant in the n-server setting, only less than an order of magnitude slower than performing plaintext inference over clouds.