Internet traffic characterisation: Third-order statistics & higher-order spectra for precise traffic modelling

Undoubtedly, the characterisation of network traffic flows is vitally important in understanding the dynamics of Internet traffic and in appropriately dimensioning network resources for network and systems management. The vast majority of modelling techniques developed for volume-based traffic profiling (based on packet and/byte counts) imply the statistical assumptions of stationarity, Gaussianity and linearity, which are often taken for granted without being explicitly validated. In this paper, we demonstrate that such properties are often not applicable due to the high fluctuations in Internet traffic, and should therefore be validated first before they are assumed. We employ Time-Frequency (TF) representations and the Hinich algorithms for validating these three modelling assumptions on real backbone and edge network traces. We show by conducting a passive, offline statistical analysis on real operational network traffic traces from both backbone and edge links that link traffic is extremely dynamic irrespective of the level of aggregation and that model characteristics vary. Subsequently, we propose the use of a representative of higher order spectra, the bispectrum, to act as a particularly suitable method for volume-based traffic profiling due to its ability to adapt to different underlying statistical assumptions, as opposed to ARIMA timeseries models that have been typically used in the literature. We demonstrate that the bispectrum, a signal processing tool that has so far been used in the area of image processing and acoustic signals, can be exploited to accurately characterise traffic volumes per transport protocol, and can therefore contribute to fine-grained network operations tasks such as application classification and anomaly detection. (C) 2018 The Author(s). Published by Elsevier B.V.