Cart and best-ortho-basis: A connection

被引:130
作者
Donoho, DL
机构
[1] Stanford Univ, Dept Stat, Stanford, CA 94305 USA
[2] Univ Calif Berkeley, Berkeley, CA 94720 USA
关键词
wavelets; anisotropic smoothness; anisotropic Haar basis; best orthogonal basis; minimax estimation; spatial adaptation; oracle inequalities;
D O I
10.1214/aos/1069362377
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
We study what we call "dyadic CART'-a method of nonparametric regression which constructs a recursive partition by optimizing a complexity penalized sum of squares, where the optimization is over all recursive partitions arising from midpoint splits. We show that the method is adaptive to unknown degrees of anisotropic smoothness. Specifically, consider the anisotropic smoothness classes of Nikol'skii, consisting of bivariate functions f(x(1), x(2)) whose finite difference of distance h in direction i is bounded in L(p) norm by Ch(delta i), i = 1, 2. W, show that dyadic CART, with an appropriate complexity penalty parameter lambda similar to sigma(2) Const.log(n), is within logarithmic terms of minimax over every anisotropic smoothness class 0 < C < infinity, 0 < delta(1), delta(2) less than or equal to 1. The proof shows that dyadic CART is identical to a certain adaptive best-ortho-basis algorithm based on the library of all anisotropic Haar bases. Then it applies empirical basis selection ideas of Donoho and Johnstone. The basis empirically selected by dyadic CART is shown to be nearly as good as a basis ideally adapted to the underlying f. The risk of estimation in an ideally adapted anisotropic Haar basis is shown to be comparable to the minimax risk over anisotropic smoothness classes. Underlying the success of this argument is harmonic analysis of anisotropic smoothness classes. We show that, for each anisotropic smoothness class, there is an anisotropic Haar basis which is a best orthogonal basis for representing that smoothness class; the basis is optimal not just within the library of anisotropic Haar bases, but among all orthogonal bases of L(2)[0, 1](2).
引用
收藏
页码:1870 / 1911
页数:42
相关论文
共 31 条
  • [1] WAVELET-LIKE BASES FOR THE FAST SOLUTION OF 2ND-KIND INTEGRAL-EQUATIONS
    ALPERT, B
    BEYLKIN, G
    COIFMAN, R
    ROKHLIN, V
    [J]. SIAM JOURNAL ON SCIENTIFIC COMPUTING, 1993, 14 (01) : 159 - 184
  • [2] [Anonymous], 1983, CLASSIFICATION REGRE
  • [3] [Anonymous], 1969, APPROXIMATION FUNCTI
  • [4] [Anonymous], 1985, PROBLEMY PEREDACHI I
  • [5] MINIMUM COMPLEXITY DENSITY-ESTIMATION
    BARRON, AR
    COVER, TM
    [J]. IEEE TRANSACTIONS ON INFORMATION THEORY, 1991, 37 (04) : 1034 - 1054
  • [6] BENNETT N, 1995, THESIS YALE U
  • [7] BIRGE L, 1994, FESTSCHRIFT LUCIEN C, P55
  • [8] ENTROPY-BASED ALGORITHMS FOR BEST BASIS SELECTION
    COIFMAN, RR
    WICKERHAUSER, MV
    [J]. IEEE TRANSACTIONS ON INFORMATION THEORY, 1992, 38 (02) : 713 - 718
  • [9] COIFMAN RR, 1994, NATO ADV SCI INST SE, V442, P363
  • [10] DEVORE RA, 1994, WAVELETS IMAGES SURF