A unified kernel sparse representation framework for supervised learning problems

For supervised learning problems, a unified kernel sparse representation framework is proposed. It is applicable to almost all supervised learners in order to look for kernel representation hypersurface, such as SVM-type or TSVM-type models. Focusing on classification and regression problems in supervised learning, three concrete sparse TSVM-type models are constructed by incorporating specific regularization terms and loss functions. Our methods involve selecting L representative points from the entire training set using the GRPS algorithm. The sparsization parameter L significantly reduces computational complexity by avoiding the need to process all training points. As a result, both the optimization problems and prediction computation costs for new instances are reduced. By comparing our sparse TSVMs with the methods based on the sparse norm regularization terms, our sparsization parameter L is more intuitional than their regularization parameter. Interestingly enough, the numerical experiments on four artificial datasets and 20 benchmark datasets demonstrate that our methods require less prediction time and exhibit better generalization ability when the sparsization parameter L is taken as a small value, i.e., L << N.