A cross-validation based estimation of the proportion of true null hypotheses

In the multiple testing context, a challenging problem is the estimation of the proportion pi(0) of true null hypotheses. A large number of estimators of this quantity rely on identifiability assumptions that either appear to be violated on real data, or can be at least relaxed. The proposed estimator (pi) over cap (0) results from density estimation by histograms, and cross-validation. Several consistency results are derived under independence. A new (plug-in) multiple testing procedure (MTP) is also described, based on the Benjamini and Hochberg procedure (BH-procedure) and the proposed estimator. This procedure is asymptotically optimal, provides the asymptotic desired false discovery rate (FDR) control, and is more powerful than the BH-procedure. The non-asymptotic behavior of (pi) over cap is finally assessed through several simulation experiments. It outperforms numerous existing estimators in usual settings, and remains accurate with "U-shape" densities where other estimators usually fail. It does not exhibit any strong sensitivity to dependence. With m block-structured dependent data, it stays reliable up to a within block correlation rho = 0.5, when m/50 blocks are used. (C) 2010 Elsevier B.V. All rights reserved.