Towards Practical Secure Privacy-Preserving Machine (Deep) Learning with Distributed Data

A methodology for practical secure privacy-preserving distributed machine (deep) learning is proposed via addressing the core issues of fully homomorphic encryption, differential privacy, and scalable fast machine learning. Considering that private data is distributed and the training data may contain directly or indirectly an information about private data, an architecture and a methodology are suggested for 1. mitigating the impracticality issue of fully homomorphic encryption (arising from large computational overhead) via very fast gate-by-gate bootstrapping and introducing a learning scheme that requires homomorphic computation of only efficient-to-evaluate functions; 2. addressing the privacy-accuracy tradeoff issue of differential privacy via optimizing the noise adding mechanism; 3. defining an information theoretic measure of privacy-leakage for the design and analysis of privacy-preserving schemes; and 4. addressing the optimal model size determination issue and computationally fast training issue of scalable and fast machine (deep) learning with an alternative approach based on variational learning. A biomedical application example is provided to demonstrate the application potential of the proposed methodology.