Comparative study on local and global strategies for confidence estimation in neural networks and extensions to improve their predictive power

The use of confidence estimation techniques on neural networks outputs plays an important role when these mathematical models are applied in many practical applications. In general, the method to provide confidence estimation is dependent on the neural network architecture, but traditionally, most popular prediction interval (PI) estimation methods are only valid under strong assumptions, which are rarely satisfied in practical real problems. In this paper, we present a comparative study of local and global strategies for PI calculations and propose novel methods in both approaches to improve the predictive power of multilayer perceptron and radial basis function neural network models when the data are heterogeneous, both in density and residual variance. We apply our methods and make comparisons in a variety of simulated and real problems.