Nonlinear System Identification Using Neural Networks Trained with Natural Gradient Descent

We use natural gradient (NG) learning neural networks (NNs) for modeling and identifying nonlinear systems with memory. The nonlinear system is comprised of a discrete-time linear filter[inline-graphic not available: see fulltext] followed by a zero-memory nonlinearity[inline-graphic not available: see fulltext]. The NN model is composed of a linear adaptive filter[inline-graphic not available: see fulltext] followed by a two-layer memoryless nonlinear NN. A Kalman filter-based technique and a search-and-converge method have been employed for the NG algorithm. It is shown that the NG descent learning significantly outperforms the ordinary gradient descent and the Levenberg-Marquardt (LM) procedure in terms of convergence speed and mean squared error (MSE) performance.