Remember the curse of dimensionality: the case of goodness-of-fit testing in arbitrary dimension

被引:19
作者
Arias-Castro, Ery [1 ]
Pelletier, Bruno [2 ]
Saligrama, Venkatesh [3 ]
机构
[1] Univ Calif San Diego, Dept Math, San Diego, CA 92103 USA
[2] Univ Rennes II, Dept Math, CNRS, IRMAR,UMR 6625, Rennes, France
[3] Boston Univ, Dept Elect & Comp Engn, Boston, MA 02215 USA
基金
美国国家科学基金会;
关键词
Curse of dimensionality; goodness-of-fit testing; minimax tests; MANIFOLD ESTIMATION; DISTRIBUTIONS; HYPOTHESES;
D O I
10.1080/10485252.2018.1435875
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
Despite a substantial literature on nonparametric two-sample goodness-of-fit testing in arbitrary dimensions, there is no mention there of any curse of dimensionality. In fact, in some publications, a parametric rate is derived. As we discuss below, this is because a directional alternative is considered. Indeed, even in dimension one, Ingster, Y.I. [(1987). Minimax testing of nonparametric hypotheses on a distribution density in the l_p metrics. Theory of Probability & Its Applications, 31(2), 333-337] has shown that the minimax rate is not parametric. In this paper, we extend his results to arbitrary dimension and confirm that the minimax rate is not only nonparametric, exhibits but also a prototypical curse of dimensionality. We further extend Ingster's work to show that the chi-squared test achieves the minimax rate. Moreover, we show that the test adapts to the intrinsic dimensionality of the data. Finally, in the spirit of Ingster, Y.I. [(2000). Adaptive chi-square tests. Journal of Mathematical Sciences, 99(2), 1110-1119], we consider a multiscale version of the chi-square test, showing that one can adapt to unknown smoothness without much loss in power.
引用
收藏
页码:448 / 471
页数:24
相关论文
共 33 条
[1]  
[Anonymous], 1951, P 2 BERKELEY S MATH
[2]  
[Anonymous], 2005, N-distances and Their Applications
[3]  
[Anonymous], 2012, P 25 INT C NEUR INF
[4]  
[Anonymous], 2004, INTERSTAT
[5]  
[Anonymous], 1959, Transactions of the American Mathematical Society, DOI [10.1090/S0002-9947-1959-0110078-1, DOI 10.1090/S0002-9947-1959-0110078-1]
[6]   Spectral clustering based on local linear approximations [J].
Arias-Castro, Ery ;
Chen, Guangliang ;
Lerman, Gilad .
ELECTRONIC JOURNAL OF STATISTICS, 2011, 5 :1537-1587
[7]   Networks of polynomial pieces with application to the analysis of point clouds and images [J].
Arias-Castro, Ery ;
Efros, Boris ;
Levi, Ofer .
JOURNAL OF APPROXIMATION THEORY, 2010, 162 (01) :94-130
[8]   Goodness of fit and homogeneity tests on the basis of N-distances [J].
Bakshaev, Aleksej .
JOURNAL OF STATISTICAL PLANNING AND INFERENCE, 2009, 139 (11) :3750-3758
[9]  
Baraud Y, 2003, ANN STAT, V31, P225
[10]  
BERLINET A., 2011, Reproducing Kernel Hilbert Spaces in Probability and Statistics