BFV, CKKS, TFHE: Which One is the Best for a Secure Neural Network Evaluation in the Cloud?

We provide clear and concise guidelines for the use of three of the most popular homomorphic cryptosystems: BFV, CKKS and TFHE. Because they are unified under the Chimera framework and it is now possible to switch a ciphertext from one cryptosystem to another, such a comparison is essential to better understand which cryptosystem to use in which use-case or for which part of a secure computation on the cloud. We do this by comparing the application of the three cryptosystems to the evaluation phase of standard feed-forward neural networks tested on the MNIST (http://yann.lecun.com/exdb/mnist/) database. We tested their application in the case where both the query and the neural network model are encrypted and in the case when only the query is encrypted. We evaluated the results obtained using the three homomorphic schemes in terms of precision, memory usage and execution time for a minimal security of 128 bits.