MAP model selection in Gaussian regression

被引:18
|
作者
Abramovich, Felix [1 ]
Grinshtein, Vadim [2 ]
机构
[1] Tel Aviv Univ, Dept Stat & Operat Res, IL-69978 Tel Aviv, Israel
[2] Open Univ Israel, Dept Math, IL-43107 Raanana, Israel
来源
基金
以色列科学基金会;
关键词
Adaptivity; complexity penalty; Gaussian linear regression; maximum a posteriori rule; minimax estimation; model selection; oracle inequality; sparsity; STATISTICAL ESTIMATION; BAYESIAN TESTIMATION;
D O I
10.1214/10-EJS573
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
We consider a Bayesian approach to model selection in Gaussian linear regression, where the number of predictors might be much larger than the number of observations. From a frequentist view, the proposed procedure results in the penalized least squares estimation with a complexity penalty associated with a prior on the model size. We investigate the optimality properties of the resulting model selector. We establish the oracle inequality and specify conditions on the prior that imply its asymptotic minimaxity within a wider range of sparse and dense settings for "nearly-orthogonal" and "multicollinear" designs.
引用
收藏
页码:932 / 949
页数:18
相关论文
共 50 条
  • [1] Model Selection for Gaussian Process Regression
    Gorbach, Nico S.
    Bian, Andrew An
    Fischer, Benjamin
    Bauer, Stefan
    Buhmann, Joachim M.
    PATTERN RECOGNITION (GCPR 2017), 2017, 10496 : 306 - 318
  • [2] Model selection for Gaussian regression with random design
    Birgé, L
    BERNOULLI, 2004, 10 (06) : 1039 - 1051
  • [3] Automatic Model Selection of the Mixtures of Gaussian Processes for Regression
    Qiang, Zhe
    Ma, Jinwen
    ADVANCES IN NEURAL NETWORKS - ISNN 2015, 2015, 9377 : 335 - 344
  • [4] Transductive Gaussian process regression with automatic model selection
    Le, Quoc V.
    Smola, Alex J.
    Gaertner, Thomas
    Altun, Yasemin
    MACHINE LEARNING: ECML 2006, PROCEEDINGS, 2006, 4212 : 306 - 317
  • [5] General model selection estimation of a periodic regression with a Gaussian noise
    Victor Konev
    Serguei Pergamenchtchikov
    Annals of the Institute of Statistical Mathematics, 2010, 62 : 1083 - 1111
  • [6] General model selection estimation of a periodic regression with a Gaussian noise
    Konev, Victor
    Pergamenchtchikov, Serguei
    ANNALS OF THE INSTITUTE OF STATISTICAL MATHEMATICS, 2010, 62 (06) : 1083 - 1111
  • [7] Model selection and signal extraction using Gaussian Process regression
    Gandrakota, Abhijith
    Lath, Amit
    Morozov, Alexandre V.
    Murthy, Sindhu
    JOURNAL OF HIGH ENERGY PHYSICS, 2023, 2023 (02)
  • [8] Model selection and signal extraction using Gaussian Process regression
    Abhijith Gandrakota
    Amit Lath
    Alexandre V. Morozov
    Sindhu Murthy
    Journal of High Energy Physics, 2023
  • [9] HISTOGRAM SELECTION IN NON GAUSSIAN REGRESSION
    Sauve, Marie
    ESAIM-PROBABILITY AND STATISTICS, 2009, 13 : 70 - 86
  • [10] Gaussian Model Selection
    Massart, Pascal
    CONCENTRATION INEQUALITIES AND MODEL SELECTION: ECOLE D'ETE DE PROBABILITES DE SAINT-FLOUR XXXIII - 2003, 2007, 1896 : 83 - 146