STOCHASTIC GRADIENT ADAPTATION UNDER GENERAL ERROR CRITERIA

In this paper we examine a family of adaptive filter algorithms of the form W(k+1) = W(k) + muf(d(k) - W(k)(t)X(k))X(k) in which f(.) is a memoryless odd-symmetric nonlinearity acting upon the error. Such algorithms are a generalization of the least-mean-square (LMS) adaptive filtering algorithm for even-symmetric error criteria. For this algorithm family, we derive general expressions for the mean and mean-square convergence of the filter coefficients for both arbitrary stochastic input data and Gaussian input data. We then provide methods for optimizing the nonlinearity to minimize the algorithm misadjustment for a given convergence rate. Using the calculus of variations, it is shown that the optimum nonlinearity to minimize misadjustment near convergence under slow adaptation conditions is independent of the statistics of the input data and can be expressed as -p'(x)/p(x), where p(x) is the probability density function or the uncorrelated plant noise. For faster adaptation under the white Gaussian input and noise assumptions, the nonlinearity is shown to be x/{1 + mulambdax2/sigma(k)2}, where lambda is the input signal power and sigma(k)2 is the conditional error power. Thus, the optimum stochastic gradient error criterion for Gaussian noise is not mean-square. Evaluating the expected behavior, it is shown that the equations governing the convergence of the nonlinear algorithm are exactly those which describe the behavior of the optimum scalar data nonlinear adaptive algorithm for white Gaussian input. Simulations verify the results for a host of noise interferences and indicate the improvement that may be achieved using non-mean-square error criteria.