THE PROBLEM OF UNDERESTIMATING THE RESIDUAL ERROR VARIANCE IN FORWARD STEPWISE REGRESSION

Under the global null hypothesis that all covariates are unrelated to the outcome variables, forward stepwise regression procedures should have the property that the probability of selecting a given variable and finding it significant at the alpha level is equal to alpha. Because of the problem of underestimating the residual error variance the actual probability can be very different from alpha. This problem becomes of practical concern when the ratio of the number of variables to the number of observations becomes greater than 0.25, and is more serious for logistic than for linear regression.