Defending ML-Based Feedback Loop System Against Malicious Adversarial Inference Attacks

Artificial Intelligence (AI) and its subset Machine Learning (ML) have been developing significantly in recent years. These technologies provide means to design and implement novel and smart applications utilizing data to be collected and processed locally or at the cloud. The data gathered can be used as a feed for training ML models able to generate predictions to be applied in fields such as construction, healthcare, military, or transportation. The hypothesis is that ML environment is not malicious, but benign. In the real world, perpetrators may try to maliciously alter the testing dataset by initiating oracle inference attacks. Oracle attacks can be categorized as ML model extraction, model inversion, or membership inference attacks. A perpetrator may chain the attacks and conduct a model extraction attack prior to the model inversion attack in order to restore ML model parameters. After extraction of the model, the perpetrator can then conduct a model inversion attack to learn the training dataset and apply evasion attack to learn a similar model. These kinds of attacks can pose a significant threat to ML utilized predictive cyber-physical system (CPS) adjusting heating, ventilation, and air conditioning (HVAC) or deceive facial recognition to gain access in facilities from smart offices to airport and data centers. In this article, we examine the concepts of adversarial machine learning (AML) and ML, AI and cybersecurity, and conduct a literature review on relevant malicious inference attack vectors. These vectors can be used in applying attacks towards ML-based smart building feedback loop systems in the cyber-physical system context. Corresponding countermeasures to prevent these attacks are examined accordingly.