SD-Transformer: A System-level Denoising Transformer for Encrypted Traffic Behavior Identification

Encrypted behavior identification is crucial in ensuring network security. Most existing solutions in this area recognize behavior by observing encrypted traffic patterns between users and applications. However, such solutions rely on features such as timing, packet sequence, and packet length, which may be affected by network fluctuations, and thus have weak generalization capabilities. In this paper, we first analyze the impact of noise on the network, such as parameters and network delays during API requests. By combining a noise-based traffic collector with an improved Transformer model, we propose a system-level denoising Transformer method for encrypted traffic behavior identification called SD-Transformer. It is able to filter system noise by utilizing an attention mechanism and targeted noise packet masking. We evaluate the performance of SD-Transformer on three datasets, i.e., ISCX-VPN, USTC-TFC, and our generated noise-containingWeb Application Traffic dataset (WEB-APP), and it achieves an accuracy of 95.97%, 93.59%, and 99.82%, respectively. Besides, compared to the state-of-the-art methods, the accuracy is increased to 96.82% (.16.0%) and 85.41% (.17.76%) on the WEB-APP dataset under different API parameters and network latency environments, respectively. Additionally, the target mask of the SD-Transformer achieves 96.45% accuracy with an improvement of 11.29% on the WEB-APP dataset with latency.