Generalization capacity of multi-class SVM based on Markovian resampling

被引:3
作者
Dong, Zijie [1 ]
Xu, Chen [2 ]
Xu, Jie [3 ]
Zou, Bin [4 ]
Zeng, Jingjing [5 ]
Tang, Yuan Yan [6 ]
机构
[1] Hubei Univ Educ, Sch Math & Econ, Wuhan 430205, Peoples R China
[2] Univ Ottawa, Dept Math & Stat, Ottawa, ON KIN 6N5, Canada
[3] Hubei Univ, Fac Comp Sci & Informat Engn, Wuhan 430062, Peoples R China
[4] Hubei Univ, Fac Math & Stat, Hubei Key Lab Appl Math, Wuhan 430062, Peoples R China
[5] Wuhan Inst Technol, Fac Math, Wuhan 430205, Peoples R China
[6] Univ Macau, Fac Sci & Technol, Macau 999078, Peoples R China
关键词
MSVM; Markovian resampling; Learning rate; Generalization bound; SUPPORT VECTOR MACHINES; CLASSIFICATION;
D O I
10.1016/j.patcog.2023.109720
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The generalization performance of "All-in-one" Multi-class SVM (AIO-MSVM) based on uniformly ergodic Markovian chain (u.e.M.c.) samples is considered. We establish the fast learning rate of AIO-MSVM al-gorithm with u.e.M.c. samples and prove that AIO-MSVM algorithm with u.e.M.c. samples is consistent. We also propose a novel AIO-MSVM algorithm based on q-times Markovian resampling (AIO-MSVM-MR), and show the numerical investigation on the learning performance of AIO-MSVM-MR based on public datasets. The experimental studies indicate that compared to the classical AIO-MSVM algorithm and other MSVM algorithms, the proposed AIO-MSVM-MR algorithm has not only smaller misclassification rate, but also less sampling and training total time. We present some discussions on the case of unbalanced training samples, the choices of q and two technical parameters, and present some explanations on the learning performance of the proposed algorithm.& COPY; 2023 Elsevier Ltd. All rights reserved.
引用
收藏
页数:13
相关论文
共 37 条
[1]   THEORY OF REPRODUCING KERNELS [J].
ARONSZAJN, N .
TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY, 1950, 68 (MAY) :337-404
[2]  
BOTTOU L, 1994, INT C PATT RECOG, P77, DOI 10.1109/ICPR.1994.576879
[3]   Multicategory classification by support vector machines [J].
Bredensteiner, EJ ;
Bennett, KP .
COMPUTATIONAL OPTIMIZATION AND APPLICATIONS, 1999, 12 (1-3) :53-79
[4]   A multi-class SVM approach based on the l1-norm minimization of the distances between the reduced convex hulls [J].
Carrasco, Miguel ;
Lopez, Julio ;
Maldonado, Sebastian .
PATTERN RECOGNITION, 2015, 48 (05) :1598-1607
[5]   The consistency of multicategory support vector machines [J].
Chen, DR ;
Xiang, DH .
ADVANCES IN COMPUTATIONAL MATHEMATICS, 2006, 24 (1-4) :155-169
[6]   Incremental learning for transductive support vector machine [J].
Chen, Haiyan ;
Yu, Ying ;
Jia, Yizhen ;
Gu, Bin .
PATTERN RECOGNITION, 2023, 133
[7]   On the algorithmic implementation of multiclass kernel-based vector machines [J].
Crammer, K ;
Singer, Y .
JOURNAL OF MACHINE LEARNING RESEARCH, 2002, 2 (02) :265-292
[8]  
Cucker F, 2002, B AM MATH SOC, V39, P1
[9]  
Dogan Ü, 2016, J MACH LEARN RES, V17
[10]   Regularization networks and support vector machines [J].
Evgeniou, T ;
Pontil, M ;
Poggio, T .
ADVANCES IN COMPUTATIONAL MATHEMATICS, 2000, 13 (01) :1-50