Self-learning activation functions to increase accuracy of privacy-preserving Convolutional Neural Networks with homomorphic encryption

被引:0
|
作者
Pulido-Gaytan, Bernardo [1 ]
Tchernykh, Andrei [1 ,2 ]
机构
[1] CICESE Res Ctr, Comp Sci Dept, Ensenada, BC, Mexico
[2] RAS, Ivannikov Inst Syst Programming, Moscow, Russia
来源
PLOS ONE | 2024年 / 19卷 / 07期
关键词
ATTACK;
D O I
10.1371/journal.pone.0306420
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
The widespread adoption of cloud computing necessitates privacy-preserving techniques that allow information to be processed without disclosure. This paper proposes a method to increase the accuracy and performance of privacy-preserving Convolutional Neural Networks with Homomorphic Encryption (CNN-HE) by Self-Learning Activation Functions (SLAF). SLAFs are polynomials with trainable coefficients updated during training, together with synaptic weights, for each polynomial independently to learn task-specific and CNN-specific features. We theoretically prove its feasibility to approximate any continuous activation function to the desired error as a function of the SLAF degree. Two CNN-HE models are proposed: CNN-HE-SLAF and CNN-HE-SLAF-R. In the first model, all activation functions are replaced by SLAFs, and CNN is trained to find weights and coefficients. In the second one, CNN is trained with the original activation, then weights are fixed, activation is substituted by SLAF, and CNN is shortly re-trained to adapt SLAF coefficients. We show that such self-learning can achieve the same accuracy 99.38% as a non-polynomial ReLU over non-homomorphic CNNs and lead to an increase in accuracy (99.21%) and higher performance (6.26 times faster) than the state-of-the-art CNN-HE CryptoNets on the MNIST optical character recognition benchmark dataset.
引用
收藏
页数:31
相关论文
共 50 条
  • [31] Evaluation of Activation Functions in Convolutional Neural Networks for Image Classification Based on Homomorphic Encryption
    Jia, Huixue
    Cai, Daomeng
    Huo, Zhilin
    Wang, Cong
    Zhang, Shibin
    Zhang, Shujun
    Li, Xiaoyu
    Yang, Shan
    PROCEEDINGS OF THE 13TH INTERNATIONAL CONFERENCE ON COMPUTER ENGINEERING AND NETWORKS, VOL III, CENET 2023, 2024, 1127 : 343 - 355
  • [32] Privacy-Preserving Video Classification with Convolutional Neural Networks
    Pentyala, Sikha
    Dowsley, Rafael
    De Cock, Martine
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139, 2021, 139
  • [33] A Survey of Deep Learning Architectures for Privacy-Preserving Machine Learning With Fully Homomorphic Encryption
    Podschwadt, Robert
    Takabi, Daniel
    Hu, Peizhao
    Rafiei, Mohammad H. H.
    Cai, Zhipeng
    IEEE ACCESS, 2022, 10 : 117477 - 117500
  • [34] Privacy-Preserving Reinforcement Learning Using Homomorphic Encryption in Cloud Computing Infrastructures
    Park, Jaehyoung
    Kim, Dong Seong
    Lim, Hyuk
    IEEE ACCESS, 2020, 8 : 203564 - 203579
  • [35] Privacy-Preserving Distributed Deep Learning via Homomorphic Re-Encryption
    Tang, Fengyi
    Wu, Wei
    Liu, Jian
    Wang, Huimei
    Xian, Ming
    ELECTRONICS, 2019, 8 (04)
  • [36] Privacy-preserving federated learning based on multi-key homomorphic encryption
    Ma, Jing
    Naas, Si-Ahmed
    Sigg, Stephan
    Lyu, Xixiang
    INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, 2022, 37 (09) : 5880 - 5901
  • [37] Privacy-Preserving federated learning in medical diagnosis with homomorphic re-Encryption
    Ku, Hanchao
    Susilo, Willy
    Zhang, Yudi
    Liu, Wenfen
    Zhang, Mingwu
    COMPUTER STANDARDS & INTERFACES, 2022, 80
  • [38] HEFactory: A symbolic execution compiler for privacy-preserving Deep Learning with Homomorphic Encryption
    Cabrero-Holgueras, Jose
    Pastrana, Sergio
    SOFTWAREX, 2023, 22
  • [39] PPFLHE: A privacy-preserving federated learning scheme with homomorphic encryption for healthcare data
    Wang, Bo
    Li, Hongtao
    Guo, Yina
    Wang, Jie
    APPLIED SOFT COMPUTING, 2023, 146
  • [40] Recent advances of privacy-preserving machine learning based on(Fully) Homomorphic Encryption
    Cheng Hong
    Security and Safety, 2025, 4 (01) : 49 - 55