BAYES STATISTICAL BEHAVIOR AND VALID GENERALIZATION OF PATTERN CLASSIFYING NEURAL NETWORKS

This letter demonstrates both theoretically and experimentally that under appropriate conditions a neural network pattern classifier generates the empirical Bayes rule optimal against the empirical distribution of the sample data which are used to train the network. In addition, it is proposed that a Bayes statistical decision approach leads naturally to a probabilistic definition of the valid generalization which a neural network can be expected to generate from a finite training sample.