Evaluation of HTTP request anomaly detection model using fastText and convolutional autoencoder

With the advent of the Internet and its close connection to people's lives, web applications have become increasingly important. To ensure that the web application is secure, a web application firewall (WAF) detects and stops attacks that exploit application vulnerabilities in communication with server applications. However, these firewalls require continuous tuning by experts with in-depth knowledge of the technologies and services provided, which may become a major obstacle to the introduction of WAF. To resolve this problem, we developed two autoencoder-based models based on an unsupervised learning model that uses only normal requests, considering the implementation and operation costs. We then evaluated the performance of the two autoencoder-based models. The first model converts a hypertext transfer protocol (HTTP) request into ASCII codes and learns their relationship in a normal request using an autoencoder. The second model generates an array of word vectors using fastText and learns using a convolutional autoencoder, which solves the problem identified in the performance evaluation of the first model where the problem was that the simple conversion to ASCII codes was not enough to distinguish between normal and anomalous requests. The two models were evaluated using the HTTP DATASET CSIC2010 dataset. The AUC for the second model was approximately 0.94 while that for the first model was approximately 0.71. This means that the second model has higher accuracy despite being an unsupervised approach, one that does not require labeled anomalous requests, and can be applied with low costs.