DPWSS: differentially private working set selection for training support vector machines

被引：0
作者：
Sun Z. [1 ,2 ]
Yang J. [1 ]
Li X. [2 ]
Zhang J. [1 ]
机构：
[1] College of Computer Science and Technology, Harbin Engineering University, Heilongjiang, Harbin
[2] College of Computer and Control Engineering, Qiqihar University, Heilongjiang, Qiqihar
来源：
Yang, Jing (yangjing@hrbeu.edu.cn) | 1600年 / PeerJ Inc.卷 / 07期
基金：
中国国家自然科学基金;
关键词：
Differential privacy; Exponential mechanism; Sequential minimal optimization; Support vector machines; Working set selection;
D O I：
10.7717/PEERJ-CS.799
中图分类号：
学科分类号：
摘要：
Support vector machine (SVM) is a robust machine learning method and is widely used in classification. However, the traditional SVM training methods may reveal personal privacy when the training data contains sensitive information. In the training process of SVMs, working set selection is a vital step for the sequential minimal optimization-type decomposition methods. To avoid complex sensitivity analysis and the influence of high-dimensional data on the noise of the existing SVM classifiers with privacy protection, we propose a new differentially private working set selection algorithm (DPWSS) in this paper, which utilizes the exponential mechanism to privately select working sets. We theoretically prove that the proposed algorithm satisfies differential privacy. The extended experiments show that the DPWSS algorithm achieves classification capability almost the same as the original non-privacy SVM under different parameters. The errors of optimized objective value between the two algorithms are nearly less than two, meanwhile, the DPWSS algorithm has a higher execution efficiency than the original non-privacy SVM by comparing iterations on different datasets. To the best of our knowledge, DPWSS is the first private working set selection algorithm based on differential privacy. © 2021. All Rights Reserved.
引用
收藏
相关论文
共 27 条
  • [1] Boser BE, Guyon I, Vapnik V., A training algorithm for optimal margin classifiers, Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 144-152, (1992)
  • [2] Burges CJC., A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 2, 2, pp. 121-167, (1998)
  • [3] Chang CC, Lin CJ., LIBSVM: a library for support vector machines, ACM Transactions on Intelligent Systems and Technology, 2, 3, (2007)
  • [4] Chaudhuri K, Monteleoni C., Privacy-preserving logistic regression, Advances in Neural Information Processing Systems, 21, pp. 289-296, (2009)
  • [5] Chaudhuri K, Monteleoni C, Sarwate AD., Differentially private empirical risk minimization, Journal of Machine Learning Research, 12, pp. 1069-1109, (2011)
  • [6] Chen PH, Fan RE, Lin CJ., A study on SMO-type decomposition methods for support vector machines, IEEE Transactions on Neural Networks, 17, 4, pp. 893-908, (2006)
  • [7] Cortes C, Vapnik V., Support vector network, Machine Learning, 20, 3, pp. 273-297, (1995)
  • [8] Dwork C., Differential privacy, Proceedings of the 33th Colloquium on Automata, Languages and Programming, pp. 1-12, (2006)
  • [9] Dwork C, McSherry F, Nissim K, Smith A., Calibrating noise to sensitivity in private data analysis, Proceedings of the 3rd Conference on Theory of Cryptography, pp. 265-284, (2006)
  • [10] Fan RE, Chen PH, Lin CJ., Working set selection using second order information for training support vector machines, Journal of Machine Learning Research, 6, 4, pp. 1889-1918, (2005)
123