Multioutput feedforward neural network selection: A Bayesian approach

A Bayesian method for the selection of multioutput feedforward neural networks based on their respective predictive capability is proposed. This paper extends, with full theoretical arguments, an approach initiated previously for the selection of single-output feedforward neural networks. As measure of the future prediction fitness, an expected utility criterion is considered which is consistently estimated by a sample-reuse computation. As opposed to classic point-prediction-based crossvalidation methods, this expected utility is defined from the logarithmic score of the neural model predictive probability density. It is shown how the advocated choice of conjugate distributions as priors for the network parameters allows consistent approximations of the respective network predictive posterior densities, in the set of competing networks. A comparison of the performance of the proposed method with those of usual selection procedures such as classic cross-validation and information-theoretic criteria, is performed first on the data of a well-known case study and then on the data of a simulated case study.