GCN-MHSA: A novel malicious traffic detection method based on graph convolutional neural network and multi-head self-attention mechanism

With the increasing size and complexity of network, network traffic becomes more and more correlated with each other, and the traditional manner of presenting network traffic in a Euclidean structure is difficult to effectively capture the correlation information of network traffic. In contrast, graph structured data has gained much attention in recent years due to its ability to represent the correlation between different traffic flows; In addition, models and algorithms related to Graph Convolution Neural network (GCN) have been used for malicious traffic detection. However, existing GCN-based malicious traffic detection methods still suffer from incomplete description of the flow-level features of network traffic, imperfect traffic correlation establishment mechanism and failure to distinguish the importance of features during model training. Based on this, this study proposes a malicious traffic detection method called GCN-MHSA based on Graph Convolutional Neural network and Multi-Head Self-Attention mechanism. Firstly, the flow-level features of network traffic are populated and more information close to the features are selected to describe the network traffic; And then, the link homogeneity is used to establish the correlations between network traffic; Moreover, multi-head self-attention mechanism is introduced in the GCN model to provide larger weight to important features; Finally, an improved GCN is used as a deep learning model to detect malicious traffic. Extensive experimental results on three publicly available network traffic datasets and a real network traffic dataset show that the proposed GCN-MHSA method performs better than five baselines in terms of detection effect and stability, with an improvement of about 2.4% in accuracy, recall and F1-measure as well as an improvement of about 2.1% in precision.