An adaptive framework for the detection of novel botnets

Detecting and disrupting botnet activities is critical for the reliability, availability and security of Internet services. However, despite many efforts in this direction, key challenges remain. These include the high computational requirements of processing large amounts of network information, the similarity between botnet and normal traffic, and the constant creation of new botnet mechanisms to bypass current detection approaches. Because of these challenges, existing detection approaches have difficulties in detecting novel botnets with high accuracy and low false positive rate. In this paper, we address this problem with an scalable and decentralized framework. Our framework creates a complete characterization of the behavior of legitimate hosts that can be used to discover previously unseen botnet traffic. Moreover, our framework dynamically adapts to changes in network traffic, and is capable of detecting novel botnets without any assumption on their architecture or protocols employed. This is crucial to nullify the constant efforts by botnet managers to adapt to current detection techniques. Through an experimental analysis using the most realistic and varied publicly available botnet dataset, we find that our framework can detect bots in a network with 1.00 TPR and 0.082 FPR or, alternatively, can detect half of the malicious hosts with a FPR as low as 0.0017. These results significantly improve the results reported by similar works in the area, with the added value of not relying on historical botnet data or specific architectures and protocols. (C) 2018 Elsevier Ltd. All rights reserved.