Multiple-v support vector regression based on spectral risk measure minimization

Statistical learning theory provides the justification of the c-insensitive loss in support vector regression, but suggests little guidance on the determination of the critical hyper-parameter c. Instead of predefining c, v-support vector regression automatically selects c by making the percent of deviations larger than c be asymptotically equal to v. In stochastic programming terminology, the goal of v-support vector regression is to minimize the conditional Value-at-Risk measure of deviations, i.e. the expectation of the larger v-percent deviations. This paper tackles the determination of the critical hyper-parameter v in v-support vector regression when the error term follows a complex distribution. Instead of one singleton v, the paper assumes v to be a combination of multiple, finite or infinite, candidate choices. Thus, the cost function becomes a weighted sum of component conditional value-at-risk measures associated with these base vs. This paper shows that this cost function can be represented with a spectral risk measure and its minimization can be reformulated to a linear programming problem. Experiments on three artificial data sets show that this multiple-v support vector regression has great advantage over the classical v-support vector regression when the error terms follow mixed polynomial distributions. Experiments on 10 real-world data sets also clearly demonstrate that this new method can achieve better performance than c-support vector regression and v-support vector regression. (C) 2012 Elsevier B.V. All rights reserved.