Poisoning attacks on machine learning models in cyber systems and mitigation strategies

Poisoning attacks on training data are becoming one of the top concerns among users of machine learning systems. The goal of such attacks is to inject a small set of maliciously mislabeled training data into the training pipeline so as to detrimentally impact a machine learning model trained on such data. Constructing such attacks for cyber applications is especially challenging due to their realizability constraints. Furthermore, poisoning mitigation techniques for such applications are also not well understood. This paper investigates techniques for realizable data poisoning availability attacks (using several cyber applications), in which an attacker can insert a set of poisoned samples at the training time with the goal of degrading the accuracy of the deployed model. We design a white-box, realizable poisoning attack that degraded the original model's accuracy by generating mislabeled samples in close vicinity of a selected subset of training points. We investigate this strategy and its modifications for key classifier architectures and provide specific implications for each of them. The paper also proposes a novel data cleaning method as a defense against such poisoning attacks. Our defense includes a diversified ensemble of classifiers, each trained on a different subset of the training set. We use the disagreement of the classifiers' predictions as a decision whether to keep a given sample in the training dataset or remove it. The results demonstrate the efficiency of this strategy with very limited performance penalty.