An Approach to Hyperparameter Optimization for the Objective Function in Machine Learning

被引:14
|
作者
Kim, Yonghoon [1 ]
Chung, Mokdong [1 ]
机构
[1] Pukyong Natl Univ, Dept Comp Engn, Pusan 48513, South Korea
基金
新加坡国家研究基金会;
关键词
bayesian optimization; gaussian process; learning rate; acauisition function; machine learning; GLOBAL OPTIMIZATION;
D O I
10.3390/electronics8111267
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In machine learning, performance is of great value. However, each learning process requires much time and effort in setting each parameter. The critical problem in machine learning is determining the hyperparameters, such as the learning rate, mini-batch size, and regularization coefficient. In particular, we focus on the learning rate, which is directly related to learning efficiency and performance. Bayesian optimization using a Gaussian Process is common for this purpose. In this paper, based on Bayesian optimization, we attempt to optimize the hyperparameters automatically by utilizing a Gamma distribution, instead of a Gaussian distribution, to improve the training performance of predicting image discrimination. As a result, our proposed method proves to be more reasonable and efficient in the estimation of learning rate when training the data, and can be useful in machine learning.
引用
收藏
页数:19
相关论文
共 50 条
  • [31] Multi-objective Hyperparameter Optimization with Performance Uncertainty
    Morales-Hernandez, Alejandro
    Van Nieuwenhuyse, Inneke
    Napoles, Gonzalo
    OPTIMIZATION AND LEARNING, OLA 2022, 2022, 1684 : 37 - 46
  • [32] Hyperparameter tuning of supervised bagging ensemble machine learning model using Bayesian optimization for estimating stormwater quality
    Mohammadreza Moeini
    Sustainable Water Resources Management, 2024, 10
  • [33] A Comparative Study of Machine Learning Models with Hyperparameter Optimization Algorithm for Mapping Mineral Prospectivity
    Lin, Nan
    Chen, Yongliang
    Liu, Haiqi
    Liu, Hanlin
    MINERALS, 2021, 11 (02) : 1 - 31
  • [34] Hyperparameter selection for dataset-constrained semantic segmentation: Practical machine learning optimization
    Boyd, Chris
    Brown, Gregory C.
    Kleinig, Timothy J.
    Mayer, Wolfgang
    Dawson, Joseph
    Jenkinson, Mark
    Bezak, Eva
    JOURNAL OF APPLIED CLINICAL MEDICAL PHYSICS, 2024, 25 (12):
  • [35] Classification of Motor Competence in Schoolchildren Using Wearable Technology and Machine Learning with Hyperparameter Optimization
    Sulla-Torres, Jose
    Gamboa, Alexander Calla
    Llanque, Christopher Avendano
    Osorio, Javier Angulo
    Carnero, Manuel Zuniga
    APPLIED SCIENCES-BASEL, 2024, 14 (02):
  • [36] Hyperparameter tuning of supervised bagging ensemble machine learning model using Bayesian optimization for estimating stormwater quality
    Moeini, Mohammadreza
    SUSTAINABLE WATER RESOURCES MANAGEMENT, 2024, 10 (02)
  • [37] Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG
    Cooney, Ciaran
    Korik, Attila
    Folli, Raffaella
    Coyle, Damien
    SENSORS, 2020, 20 (16) : 1 - 22
  • [38] An Iterative Machine Learning Approach to Informative Performance Reporting in Dynamic Multi-Objective Optimization
    Herring, Daniel
    Kirley, Michael
    Yao, Xin
    PROCEEDINGS OF THE 2023 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE COMPANION, GECCO 2023 COMPANION, 2023, : 367 - 370
  • [39] Efficient hyperparameter-tuned machine learning approach for estimation of supercapacitor performance attributes
    Ahmed, Syed Ishtiyaq
    Radhakrishnan, Sreevatsan
    Nair, Binoy B.
    Thiruvengadathan, Rajagopalan
    JOURNAL OF PHYSICS COMMUNICATIONS, 2021, 5 (11):
  • [40] Multi-Objective Optimization of Sustainable Epoxy Resin Systems through Bayesian Optimization and Machine Learning
    Albuquerque, Rodrigo Q.
    Rothenhaeusler, Florian
    Groebel, Philipp
    Ruckdaeschel, Holger
    ACS APPLIED ENGINEERING MATERIALS, 2023, 1 (12): : 3298 - 3308