Heteroscedastic kernel ridge regression

被引:36
作者
Cawley, GC [1 ]
Talbot, NLC
Foxall, RJ
Dorling, SR
Mandic, DP
机构
[1] Univ E Anglia, Sch Comp Sci, Norwich NR4 7TJ, Norfolk, England
[2] Univ E Anglia, Sch Environm Sci, Norwich NR4 7TJ, Norfolk, England
[3] Univ London Imperial Coll Sci Technol & Med, Dept Elect & Elect Engn, London SW7 2BT, England
关键词
kernel methods; non-linear regression; heteroscedasticity;
D O I
10.1016/j.neucom.2004.01.005
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper we extend a form of kernel ridge regression (KRR) for data characterised by a heteroscedastic (i.e. input dependent variance) Gaussian noise process, introduced in Foxall et al. (in: Proceedings of the European Symposium on Artificial Neural Networks (ESANN-2002), Bruges, Belgium, April 2002, pp. 19-24). It is shown that the proposed heteroscedastic kernel ridge regression model can give a more accurate estimate of the conditional mean of the target distribution than conventional KRR and also provides an indication of the spread of the target distribution (i.e. predictive error bars). The leave-one-out cross-validation estimate of the conditional mean is used in fitting the model of the conditional variance in order to overcome the inherent bias in maximum likelihood estimates of the variance. The benefits of the proposed model are demonstrated on synthetic and real-world benchmark data sets and for the task of predicting episodes of poor air quality in an urban environment. (C) 2004 Elsevier B.V. All rights reserved.
引用
收藏
页码:105 / 124
页数:20
相关论文
共 23 条
[1]  
[Anonymous], [No title captured]
[2]  
[Anonymous], P INT C COMP LEARN T
[3]   THEORY OF REPRODUCING KERNELS [J].
ARONSZAJN, N .
TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY, 1950, 68 (MAY) :337-404
[4]   AN INVERSE MATRIX ADJUSTMENT ARISING IN DISCRIMINANT ANALYSIS [J].
BARTLETT, MS .
ANNALS OF MATHEMATICAL STATISTICS, 1951, 22 (01) :107-111
[5]  
Bishop C. D., 1995, Journal of Essential Oil Research, V7, P641
[6]  
Bishop C. M., 1996, Neural networks for pattern recognition
[7]  
CAWLEY GC, 2003, P EUR S ART NEUR NET, P209
[8]  
Cook R. D., 1982, Residuals and influence in regression
[9]  
Cristianini N., 2000, Intelligent Data Analysis: An Introduction, DOI 10.1017/CBO9780511801389
[10]  
FOXALL RJ, 2002, P EUR S ART NEUR NET, P19