Robust Least-Squares Support Vector Machine Using Probabilistic Inference

The least-square support vector machine (LS-SVM) has been deeply studied in the machine-learning field and widely applied on a great deal of occasions. A disadvantage is that it is less efficient in dealing with the non-Gaussian noise. In this article, a novel probabilistic LS-SVM is proposed to enhance the modeling reliability even data contaminated by the non-Gaussian noise. The stochastic effect of noise on the kernel function and the regularization parameter is first analyzed and estimated. On the basis of this, a new objective function is constructed under a probabilistic sense. A probabilistic inference method is then developed to construct the distribution of the model parameter, including distribution estimation of both the kernel function and the regularization parameter from data. Using this distribution information, a solving strategy is then developed for this new objective function. Different from the original LS-SVM that uses a deterministic scenario approach to gain the model, the proposed method builds the distribution relation between the model and noise and makes use of this distribution information in the process of modeling; thus, it is more robust for modeling of noise data. The effectiveness of the proposed probabilistic LS-SVM is demonstrated by using both artificial and real cases.