Adversary Aware Continual Learning

被引:1
作者
Umer, Muhammad [1 ]
Polikar, Robi [2 ]
机构
[1] Fairleigh Dickinson Univ, Gildart Haase Sch Comp Sci & Engn GHSCSE, Teaneck, NJ 07666 USA
[2] Rowan Univ, Dept Elect & Comp Engn, Glassboro, NJ 08028 USA
关键词
Continuing education; Training; Computational modeling; Data models; Predictive models; Incremental learning; Fake news; Sequential analysis; Continual (incremental) learning; catastrophic forgetting; misinformation; false memory; backdoor poisoning attack; NEURAL-NETWORKS;
D O I
10.1109/ACCESS.2024.3455090
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Continual learning approaches are useful to help a model learn new information or new tasks sequentially, while also retaining the previously acquired information. However, such approaches are known to be extremely vulnerable to the adversarial backdoor attacks, where an intelligent adversary can introduce small amount of misinformation in the form of imperceptible backdoor pattern during training to cause deliberate forgetting of a specific task or class at test time. In this work, we propose a novel defensive framework to counter such an attack and use the attacker's primary strength - hiding the backdoor pattern by making it imperceptible to humans - against itself. To do so we train the model to learn a random perceptible pattern as decoy data during training with the deliberate intent to overpower the attacker's imperceptible pattern. We demonstrate the effectiveness of the proposed defensive mechanism through various commonly used replay-based (both generative and exact replay-based) continual learning algorithms using continual learning benchmark variants of CIFAR-10, CIFAR-100, and MNIST datasets. We show that the proposed defensive framework considerably improves the robustness of continual learning algorithms without having any knowledge of the attacker's target task, attacker's target class, shape, size, and location of the attacker's backdoor pattern. Moreover, our defensive framework does not depend on the underlying continual learning algorithm and does not rely on detecting the attack samples (to subsequently remove them from further consideration), but instead attempts to correctly classify even the attack samples and thus ensuring robustness in continual learning models. We refer to the proposed defensive framework as adversary aware continual learning (AACL).
引用
收藏
页码:126108 / 126121
页数:14
相关论文
共 50 条
[1]  
Bai T, 2021, Arxiv, DOI arXiv:2102.01356
[2]  
Bradski G., 2000, Dr. Dobb's J. Softw. Tools, V3, P1
[3]  
Buzzega P, 2020, Arxiv, DOI [arXiv:2004.07211, 10.48550/arXiv.2004.07211, DOI 10.48550/ARXIV.2004.07211]
[4]  
Chakraborty A, 2018, Arxiv, DOI arXiv:1810.00069
[5]  
Chan AL, 2019, Arxiv, DOI arXiv:1911.08040
[6]  
Chen B, 2019, P SAFEAI AAAI
[7]  
De Lange M, 2021, Arxiv, DOI arXiv:1909.08383
[8]   Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems [J].
Doan, Bao Gia ;
Abbasnejad, Ehsan ;
Ranasinghe, Damith C. .
36TH ANNUAL COMPUTER SECURITY APPLICATIONS CONFERENCE (ACSAC 2020), 2020, :897-912
[9]   STRIP: A Defence Against Trojan Attacks on Deep Neural Networks [J].
Gao, Yansong ;
Xu, Change ;
Wang, Derui ;
Chen, Shiping ;
Ranasinghe, Damith C. ;
Nepal, Surya .
35TH ANNUAL COMPUTER SECURITY APPLICATIONS CONFERENCE (ACSA), 2019, :113-125
[10]   NONLINEAR NEURAL NETWORKS - PRINCIPLES, MECHANISMS, AND ARCHITECTURES [J].
GROSSBERG, S .
NEURAL NETWORKS, 1988, 1 (01) :17-61