Using Automatic Programming to Improve Gradient Boosting for Classification

被引:0
作者
Olsson, Roland [1 ]
Acharya, Shubodha [1 ]
机构
[1] Ostfold Univ Coll, Halden, Ostfold, Norway
来源
ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, ICAISC 2022, PT I | 2023年 / 13588卷
关键词
Machine learning; Gradient boosting; XGBoost; LightGBM; CatBoost; AutoML; Hyperparameters; Automatic programming; Automatic design of algorithms through evolution; Meta machine learning;
D O I
10.1007/978-3-031-23492-7_21
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we present our new and automatically tuned gradient boosting software, Classifium GB, which beats its closest competitor, H2O, for all datasets that we ran. The primary reason that we found it easy to develop Classifium GB is that we employed meta machine learning, based on evolution, to automatically program its most important parts. Gradient boosting is often the most accurate classification algorithm for tabular data and quite popular in machine learning competitions. However, its practical use has been hampered by the need to skilfully tune many hyperparameters in order to achieve the best accuracy. Classifium GB contains novel regularization methods and has automatic tuning of all regularization parameters. We show that Classifium GB gives better accuracy than another automatically tuned algorithm, H2O, and often also outperforms manually tuned algorithms such as XGBoost, LightGBM and CatBoost even if the tuning of these is done with exceptional care and uses huge computational resources. Thus, our new Classifium GB algorithm should rapidly become the preferred choice for practically any tabular dataset. It is quite easy to use and even say Random Forest or C5.0 require more skilled users. The primary disadvantage is longer run time.
引用
收藏
页码:242 / 253
页数:12
相关论文
共 8 条