Statistical development and evaluation of microarray gene expression data filters

Filtering is a common practice used to simplify the analysis of microarray data by removing from subsequent consideration probe sets believed to be unexpressed. The m/ n filter, which is widely used in the analysis of Affymetrix data, removes all probe sets having fewer than m present calls among a set of n chips. The m/ n filter has been widely used without considering its statistical properties. The level and power of the m/ n filter are derived. Two alternative filters, the pooled p- value filter and the error- minimizing pooled p- value filter are proposed. The pooled p- value filter combines information from the present - absent p- values into a single summary p- value which is subsequently compared to a selected significance threshold. We show that the pooled p- value filter is the uniformly most powerful statistical test under a reasonable beta model and that it exhibits greater power than the m/ n filter in all scenarios considered in a simulation study. The error- minimizing pooled p- value filter compares the summary p- value with a threshold determined to minimize a total- error criterion based on a partition of the distribution of all probes' summary p- values. The pooled p- value and error- minimizing pooled p- value filters clearly perform better than the m/ n filter in a case- study analysis. The case- study analysis also demonstrates a proposed method for estimating the number of differentially expressed probe sets excluded by filtering and subsequent impact on the final analysis. The filter impact analysis shows that the use of even the best filter may hinder, rather than enhance, the ability to discover interesting probe sets or genes. S- plus and R routines to implement the pooled p- value and error- minimizing pooled p- value filters have been developed and are available from www. stjuderesearch. org/ depts/ biostats/ index. html.