Stepwise selection of covariates for adaptive neural networks using bootstrap

When modeling or predicting biological phenomena, one often needs to select the most important independent covariates from a larger superset of equally meaningful variables. In statistics, the procedure of stepwise regression has been known for quite some time. In the realm of artificial neural networks, however, no similar mechanism exists. Although often neural networks learn to ignore insignificant parameters during the learning phase by setting the corresponding weights to zero, the problem of selecting appropriate inputs early on is very important, especially in cases where the initial set of available parameters is so large that the network's adaptive training is severely impeded by the size of the input vector. This paper introduces a novel methodology allowing to reduce the number of covariates a neural network has to consider by ranking the estimated error contribution of each potential variable. Error components of input parameters are estimated by a hybrid bootstrap-neural net mechanism, allowing us to use their empirical distributions for selecting only variables that carry significant enough weight to be included in the covariate vector. This methodology was tested on clinical data comprising ten years of follow-up of artificial heart valve implants. Based on preoperative information, the neural system predicted various complications, and the new technique has been applied for selecting important patient characteristics for training the network.