HAXSS: Hierarchical Reinforcement Learning for XSS Payload Generation

Web application vulnerabilities are an ongoing problem that current black-box techniques and scanners do not entirely solve, suffering in particular from a lack of payload diversity that prevents them from capturing the long tail of vulnerabilities caused by uncommon sanitisation mistakes. In order to increase the diversity of payloads that can be automatically generated in a black-box fashion, we develop a hierarchical reinforcement learning approach where agents focus separately on the tasks of escaping the current context, and evading sanitisation. We implement this in an end-to-end prototype we call HAXSS. We compare our approach against a number of state-of-the-art black-box scanners on a new micro-benchmark for XSS payload generation, and on a macro-benchmark of established vulnerable web applications. HAXSS outperforms the other scanners on both benchmarks, identifying 131 vulnerabilities (a 20% improvement over the closest scanner), reporting 0 false positives. Finally, we demonstrate that our approach is practically useful, as HAXSS re-discovers 4 existing CVEs and discovers 5 new CVEs in 3 production-grade web applications.