Comparison of statistical indices using third order statistics for nonlinearity detection

In this paper we discuss the efficiency of nonlinearity indices based on higher order statistics in order to detect nonlinearities in an observed signal, the signal being the output of a transmission channel (possibly nonlinear) the input of which is not accessible. Nonlinearity detection is the first step of nonlinearity analysis, this step being followed by nonlinearity location of the nonlinear components (in the Fourier domain) and quantification of these components. The main results reported in this paper are, first, a systematic survey of the robustness of hypothesis testing for each index and, second, the derivation of indices which neither involve the ratio of estimated quantities (such as bicoherence) nor phase unwrapping (such as the bicepstrum). The robustness of hypothesis testing is verified by calculating type II error probability (i.e. the error of declaring that the time series has been produced by a linear system while produced by a nonlinear one). To calculate this error, the observed time series is assumed to be the output of a second-order Volterra model driven by a Gaussian distributed noise. Obviously, the assumption of such a model might seem restrictive, but the results obtained allow us to draw some definitive conclusions about the robustness of the indices presented. The calculation is performed first from a theoretical spectrum and bispectrum and, second, from estimated indices. These indices are estimated from linear and nonlinear time series having the same spectrum. The estimation of the type II error probability on estimated indices allows the verification of the assumptions used to derive the theoretical index probability density function. (C) 2003 Elsevier B.V. All rights reserved.