STIXGEN - A novel framework for Automatic Generation of Structured Cyber Threat Information

A large number of Advanced Persistent Threats (APTs) are being launched by nation-states, organizations and individuals within and across borders. It has been observed that APTs launched against an organization subsequently succeeded with high probability against other similar organizations. Therefore, it has become a need of the time that organizations accumulate and share Cyber Threat Information (CTI) with peers in a structured form for timely prevention and recovery of an attack. Although a large volume of cyber threat data is available on different security blogs, however this data is mostly distributed and unstructured. Presently, there is a lack of easy to use frameworks, which produce and share CTI in a structured form. Furthermore, publicly available structured data is sparse and is mostly redundant, irrelevant and erroneous. Ironically, no method has yet been devised to generate the distinct, meaningful and error-free structured data from text. In this regard, we used the standard "Structured Threat Information eXpression (STIX)". Although, STIX is a comprehensive effort, it is slow in adoption. This is due to a largely manual STIX generation process, which is naturally difficult and produces errors. We take all these deficits as a barrier in STIX utilization and these shortcomings have become a motivation for our research work. We not only proposed the STIXGEN framework, but also developed its prototype for a proof of concept. We perform evaluation of our proposed solution in terms of accuracy and effectiveness. At first, we collected different text reports, generated their STIXs via online tools and by using STIXGEN, then we compared and shared their results with domain experts. It was found that our proposed solution's results are better than other tools and are distinct, threat relevant, and error-free. Subsequently, we presented a comparative analysis of the features provided by different STIX generator tools. At the end, we provide a comprehensive STIX dataset of APTs launched against renowned industries on github, so that researchers and analysts can use it for their research.