Towards Practical Privacy-Preserving Solution for Outsourced Neural Network Inference

When neural network model and data are out-sourced to a cloud server for inference, it is desired to preserve the privacy of the model/data as the involved parties (i.e., cloud server, and model/data providing clients) may not trust mutually. Solutions have been proposed based on multi-party computation, trusted execution environment (TEE) and leveled or fully homomorphic encryption (LHE or FHE), but they all have limitations that hamper practical application. We propose a new framework based on integration of LHE and TEE, which enables collaboration among mutually-untrusted three parties, while minimizing the involvement of resource-constrained TEE but fully utilizing the untrusted but resource-rich part of server. We also propose a generic and efficient LHE-based inference scheme, along with optimizations, as an important performance-determining component of the framework. We implemented and evaluated the proposed scheme on a moderate platform, and the evaluations show that, our proposed system is applicable and scalable to various settings, and it has better or comparable performance when compared with the state-of-the-art solutions which are more restrictive in applicability and scalability.