Machine Learning Security: Threats, Countermeasures, and Evaluations

被引:102
作者
Xue, Mingfu [1 ]
Yuan, Chengxiang [1 ]
Wu, Heyi [2 ]
Zhang, Yushu [1 ]
Liu, Weiqiang [3 ]
机构
[1] Nanjing Univ Aeronaut & Astronaut, Coll Comp Sci & Technol, Nanjing 210016, Peoples R China
[2] Nanjing Upsec Network Secur Technol Res Inst Co L, Nanjing 211100, Peoples R China
[3] Nanjing Univ Aeronaut & Astronaut, Coll Elect & Informat Engn, Nanjing 210016, Peoples R China
基金
中国国家自然科学基金;
关键词
Machine learning; Security; Data models; Machine learning algorithms; Training; Training data; Prediction algorithms; Artificial intelligence security; poisoning attacks; backdoor attacks; adversarial examples; privacy-preserving machine learning; POISONING ATTACKS; DEFENSES;
D O I
10.1109/ACCESS.2020.2987435
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Machine learning has been pervasively used in a wide range of applications due to its technical breakthroughs in recent years. It has demonstrated significant success in dealing with various complex problems, and shows capabilities close to humans or even beyond humans. However, recent studies show that machine learning models are vulnerable to various attacks, which will compromise the security of the models themselves and the application systems. Moreover, such attacks are stealthy due to the unexplained nature of the deep learning models. In this survey, we systematically analyze the security issues of machine learning, focusing on existing attacks on machine learning systems, corresponding defenses or secure learning techniques, and security evaluation methods. Instead of focusing on one stage or one type of attack, this paper covers all the aspects of machine learning security from the training phase to the test phase. First, the machine learning model in the presence of adversaries is presented, and the reasons why machine learning can be attacked are analyzed. Then, the machine learning security-related issues are classified into five categories: training set poisoning; backdoors in the training set; adversarial example attacks; model theft; recovery of sensitive training data. The threat models, attack approaches, and defense techniques are analyzed systematically. To demonstrate that these threats are real concerns in the physical world, we also reviewed the attacks in real-world conditions. Several suggestions on security evaluations of machine learning systems are also provided. Last, future directions for machine learning security are also presented.
引用
收藏
页码:74720 / 74742
页数:23
相关论文
共 50 条
[41]   Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning [J].
Cina, Antonio Emanuele ;
Grosse, Kathrin ;
Demontis, Ambra ;
Vascon, Sebastiano ;
Zellinger, Werner ;
Moser, Bernhard A. ;
Oprea, Alina ;
Biggio, Battista ;
Pelillo, Marcello ;
Roli, Fabio .
ACM COMPUTING SURVEYS, 2023, 55 (13S)
[42]   Security Threats from Bitcoin Wallet Smartphone Applications: Vulnerabilities, Attacks, and Countermeasures [J].
Hu, Yiwen ;
Wang, Sihan ;
Tu, Guan-Hua ;
Xiao, Li ;
Xie, Tian ;
Lei, Xinyu ;
Li, Chi-Yu .
PROCEEDINGS OF THE ELEVENTH ACM CONFERENCE ON DATA AND APPLICATION SECURITY AND PRIVACY (CODASPY '21), 2021, :89-100
[43]   Security Threats and Possible Countermeasures in IoT Applications Covering Different Industry Domains [J].
Samaila, Musa G. ;
Sequeiros, Joao B. F. ;
Freire, Mario M. ;
Inacio, Pedro R. M. .
13TH INTERNATIONAL CONFERENCE ON AVAILABILITY, RELIABILITY AND SECURITY (ARES 2018), 2019,
[44]   Security and Privacy Threats to On-Chip Non-Volatile Memories and Countermeasures [J].
Ghosh, Swaroop ;
Khan, Mohammad Nasim Imtiaz ;
De, Asmit ;
Jang, Jae-Won .
2016 IEEE/ACM INTERNATIONAL CONFERENCE ON COMPUTER-AIDED DESIGN (ICCAD), 2016,
[45]   Trustworthy federated learning: privacy, security, and beyond [J].
Chen, Chunlu ;
Liu, Ji ;
Tan, Haowen ;
Li, Xingjian ;
Wang, Kevin I-Kai ;
Li, Peng ;
Sakurai, Kouichi ;
Dou, Dejing .
KNOWLEDGE AND INFORMATION SYSTEMS, 2025, 67 (03) :2321-2356
[46]   A Weighted Machine Learning-Based Attacks Classification to Alleviating Class Imbalance [J].
Chkirbene, Zina ;
Erbad, Aiman ;
Hamila, Ridha ;
Gouissem, Ala ;
Mohamed, Amr ;
Guizani, Mohsen ;
Hamdi, Mounir .
IEEE SYSTEMS JOURNAL, 2021, 15 (04) :4780-4791
[47]   Machine Learning Security Against Data Poisoning: Are We There Yet? [J].
Cina, Antonio Emanuele ;
Grosse, Kathrin ;
Demontis, Ambra ;
Biggio, Battista ;
Roli, Fabio ;
Pelillo, Marcello .
COMPUTER, 2024, 57 (03) :26-34
[48]   Applications of Machine Learning in Hardware Security [J].
Halak, Basel ;
Mispan, Mohd Syafiq .
2022 2ND INTERNATIONAL CONFERENCE OF SMART SYSTEMS AND EMERGING TECHNOLOGIES (SMARTTECH 2022), 2022, :212-213
[49]   Machine Learning for Optical Network Security Monitoring: A Practical Perspective [J].
Furdek, Marija ;
Natalino, Carlos ;
Lipp, Fabian ;
Hock, David ;
Di Giglio, Andrea ;
Schiano, Marco .
JOURNAL OF LIGHTWAVE TECHNOLOGY, 2020, 38 (11) :2860-2871
[50]   Security for the Metaverse: Blockchain and Machine Learning Techniques for Intrusion Detection [J].
Truong, Vu Tuan ;
Le, Long Bao .
IEEE NETWORK, 2024, 38 (05) :204-212