Boosting-Based Semi-Supervised AUC Optimization: Theory and Algorithm

被引：0

作者：

Yang Z.-Y. ^{[1
]}

Xu Q.-Q. ^{[2
]}

He Y. ^{[3
]}

Cao X.-C. ^{[4
]}

Huang Q.-M. ^{[1
,2
,5
,6
]}

机构：

[1] School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing

[2] Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing

[3] Alibaba Turing Security Lab, Beijing

[4] State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing

[5] Key Laboratory of Big Data Mining and Knowledge Management(BDKM), University of Chinese Academy of Sciences, Beijing

[6] Peng Cheng Laboratory, Shenzhen

来源：

Jisuanji Xuebao/Chinese Journal of Computers | 2022年 / 45卷 / 08期

基金：

中国国家自然科学基金;

关键词：

AUC optimization; Boosting; Ensemble learning; Rademacher complexity; Semi-supervised learning;

D O I：

10.11897/SP.J.1016.2022.01598

中图分类号：

学科分类号：

摘要：

Area Under the ROC Curve(AUC) is a standard evaluation metric for a wide range of tasks such as class-imbalance classification and bipartite ranking. This paper focuses on the semi-supervised AUC optimization problem. Most existing methods only adopt single-model-based methods, while rarely taking into account the benefit of combine multiple models. To address this issue, this paper studies the problem of how to effectively ensemble a series of semi-supervised AUC optimization methods. Specifically, we propose a boosting-based semi-supervised AUC optimization method. On top of this, we provide an acceleration strategy based on a weight decoupling strategy to reduce the time and space complexity. Moreover, we theoretically prove that the proposed algorithm has an exponential convergence rate with respect to the number of weak learners. Meanwhile, we provide a generalization error bound of the proposed method, and further prove that increasing the number of weak learners could improve the performance on the training set without the cost of a significant overfitting effect. Finally, we evaluate our proposed framework on 16 benchmark datasets. Experimental results show that the proposed algorithm outperforms all the competitors with a significance level of 0.05, and achieves a 0.9%-11.28% performance gain in average. © 2022, Science Press. All right reserved.

引用

页码：1598 / 1617

页数：19

共 34 条

[1] Fawcett T., An introduction to ROC analysis, Pattern Recognition Letters, 27, 8, pp. 861-874, (2006)
[2] Hand D J, Till R J., A simple generalisation of the area under the ROC curve for multiple class classification problems, Machine Learning, 45, 2, pp. 171-186, (2001)
[3] Hao H, Fu H, Xu Y, Et al., Open-narrow-synechiae anterior chamber angle classification in AS-OCT sequences, CoRR, (2020)
[4] Liu C, Zhong Q, Ao X, Et al., Fraud transactions detection via behavior tree with local intention calibration, Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3035-3043, (2020)
[5] Chen Y, Chen B, He X, Et al., λOpt: Learn to regularize recommender models in finer levels, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 978-986, (2019)
[6] Dai L, Yin Y, Qin C, Et al., Enterprise cooperation and competition analysis with a sign-oriented preference network, Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 774-782, (2020)
[7] Cortes C, Mohri M., AUC optimization vs. error rate minimization, Advances in Neural Information Processing Systems, pp. 313-320, (2003)
[8] Alan A H, Raskutti B., Optimising area under the ROC curve using gradient descent, Proceedings of the 21st International Conference on Machine Learning, pp. 49-56, (2004)
[9] Calders T, Jaroszewicz S., Efficient AUC optimization for classification, Proceedings of the European Conference on Principles of Data Mining and Knowledge Discovery, pp. 42-53, (2007)
[10] Freund Y, Iyer R, Schapire R E, Et al., An efficient boosting algorithm for combining preferences, Journal of Machine Learning Research, 4, pp. 933-969, (2003)

← 1 2 3 4 →