Nonparametric distributed learning under general designs

被引:5
作者
Liu, Meimei [1 ,2 ]
Shang, Zuofeng [1 ,2 ]
Cheng, Guang [3 ]
机构
[1] Virginia Tech, Dept Stat, Blacksburg, VA 24061 USA
[2] NJIT, Dept Math Sci, Newark, NJ 07102 USA
[3] Purdue Univ, Dept Stat, W Lafayette, IN 47906 USA
来源
ELECTRONIC JOURNAL OF STATISTICS | 2020年 / 14卷 / 02期
关键词
Computational limit; divide and conquer; kernel ridge regression; minimax optimality; nonparametric testing; MINIMAX-OPTIMAL RATES; REGRESSION; THEOREM;
D O I
10.1214/20-EJS1733
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
This paper focuses on the distributed learning in nonparametric regression framework. With sufficient computational resources, the efficiency of distributed algorithms improves as the number of machines increases. We aim to analyze how the number of machines affects statistical optimality. We establish an upper bound for the number of machines to achieve statistical minimax in two settings: nonparametric estimation and hypothesis testing. Our framework is general compared with existing work. We build a unified frame in distributed inference for various regression problems, including thin-plate splines and additive regression under random design: univariate, multivariate, and diverging-dimensional designs. The main tool to achieve this goal is a tight bound of an empirical process by introducing the Green function for equivalent kernels. Thorough numerical studies back theoretical findings.
引用
收藏
页码:3070 / 3102
页数:33
相关论文
共 31 条
  • [1] [Anonymous], 2018, J MACHINE LEARNING R
  • [2] Local Rademacher complexities
    Bartlett, PL
    Bousquet, O
    Mendelson, S
    [J]. ANNALS OF STATISTICS, 2005, 33 (04) : 1497 - 1537
  • [3] Cucker F, 2002, B AM MATH SOC, V39, P1
  • [4] A CENTRAL-LIMIT-THEOREM FOR GENERALIZED QUADRATIC-FORMS
    DEJONG, P
    [J]. PROBABILITY THEORY AND RELATED FIELDS, 1987, 75 (02) : 261 - 277
  • [5] Eggermont PPB, 2009, MAXIMUM PENALIZED LI, V2
  • [6] Generalized likelihood ratio statistics and Wilks phenomenon
    Fan, JQ
    Zhang, CM
    Zhang, J
    [J]. ANNALS OF STATISTICS, 2001, 29 (01) : 153 - 193
  • [7] Gine E., 2016, Mathematical foundations of infinite-dimensional statistical models, V40
  • [8] Gu C, 2013, SPRINGER SER STAT, V297, P1, DOI 10.1007/978-1-4614-5369-7
  • [9] Ingster Y., 2012, Nonparametric Goodness-of-Fit Testing Under Gaussian Models, V169
  • [10] Ingster Yu.I., 1993, Math. Methods Statist, V2, P85