A hybridization of Bayesian and neural learning

In this paper, we present a method that combines Bayesian learning, a statistical technique, with the High Order Network with Exponential SynapTic links (HONEST) network, a high order neural network with the property that the mapping embodied by the network can always be described by a polynomial-like equation. When the distribution of features in a classification task is known, statistical methods can often produce better performance than backward error propagation networks. However, if the feature distribution is not known precisely, then the performance of a statistical method will depend on how well the assumed distribution matches the actual distribution. In the hybrid method that we present here, we first use Bayesian learning, with an assumption of a multivariate normal distribution, to learn a quadratic polynomial mapping and then use the HONEST network to improve, or fine-tune, this mapping through backward error propagation. (C) 2002 Elsevier Science B.V. All rights reserved.