BIASES IN BALANCED EXPERIMENTS WITH UNCONTROLLED RANDOM FACTORS

Ignoring an uncontrolled source of random variation or pooling it with residual error (equivalent to ignoring it), or assuming that a factor has fixed effects when an assumption of randomness is more correct, can lead to severe bias in tests of the primary fixed effects. The bias causes overconfidence that observed differences in means do, in fact, reflect population differences. More correct, statistically, bias causes the frequency of false-positive inferences to exceed (often, grossly) the nominal rate of type I error, and they cause standard errors to misrepresent the precision of sample means. Biases in analysis of mixed cross-classified models and partially-hierarchical models are illustrated with five examples, one each from veterinary physiology, poultry genetics, sensory evaluation of meat, swine feeding, and cattle breeding.