Size performance of some tests in one-way ANOVA

Tsui and Weerahandi (1989) introduced the notion of generalized p-values and since then this idea is used to solve many statistical testing problems. Heteroskedasticity is one of the major practical problems encountered in ANOVA problems. To compare the means of several groups under heteroskedasticity approximate tests are used in the literature. Weerahandi (1995a) introduced a test using the notion of generalized p-values for comparing the means of several populations when the variances are not equal. This test is referred to as a generalized F-test. In this paper we compare the size performance of the Generalized F-test and four other widely used procedures: the Classical F-test for ANOVA, the F-test obtained by the weighted least-squares to adjust for heteroskedasticity, the Brown-Forsythe-test, and the Welch-test. The comparison is based on a simulation study of size performance of tests applied to the balanced one-way model. The intended level of the tests is set at 0.05. While the Generalized F-test was found to have size not exceeding the intended level, as heteroskedasticity becomes severe the other tests were found to have poor size performance. With mild heteroskedasticity the Welch-test and the classical ANOVA F-test have the intended level, and the Welch-test was found to perform better than the latter. Widely used (due to computational convenience) weighted F-test was found to have very serious size problems. The size advantage of the generalized F-test was also found to be robust even under severe deviations from the assumption of normality.