An Approach to Hyperparameter Optimization for the Objective Function in Machine Learning

被引:14
|
作者
Kim, Yonghoon [1 ]
Chung, Mokdong [1 ]
机构
[1] Pukyong Natl Univ, Dept Comp Engn, Pusan 48513, South Korea
基金
新加坡国家研究基金会;
关键词
bayesian optimization; gaussian process; learning rate; acauisition function; machine learning; GLOBAL OPTIMIZATION;
D O I
10.3390/electronics8111267
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In machine learning, performance is of great value. However, each learning process requires much time and effort in setting each parameter. The critical problem in machine learning is determining the hyperparameters, such as the learning rate, mini-batch size, and regularization coefficient. In particular, we focus on the learning rate, which is directly related to learning efficiency and performance. Bayesian optimization using a Gaussian Process is common for this purpose. In this paper, based on Bayesian optimization, we attempt to optimize the hyperparameters automatically by utilizing a Gamma distribution, instead of a Gaussian distribution, to improve the training performance of predicting image discrimination. As a result, our proposed method proves to be more reasonable and efficient in the estimation of learning rate when training the data, and can be useful in machine learning.
引用
收藏
页数:19
相关论文
共 50 条
  • [41] On Hyperparameter Optimization of Machine Learning Methods Using a Bayesian Optimization Algorithm to Predict Work Travel Mode Choice
    Aghaabbasi, Mahdi
    Ali, Mujahid
    Jasinski, Michal
    Leonowicz, Zbigniew
    Novak, Tomas
    IEEE ACCESS, 2023, 11 : 19762 - 19774
  • [42] Arabic Sentiment Analysis for ChatGPT Using Machine Learning Classification Algorithms: A Hyperparameter Optimization Technique
    Nasayreh, Ahmad
    Al Mamlook, Rabia Emhamed
    Samara, Ghassan
    Gharaibeh, Hasan
    Aljaidi, Mohammad
    Alzu'bi, Dalia
    Al-Daoud, Essam
    Abualigah, Laith
    ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING, 2024, 23 (03)
  • [43] Metaheuristic-based hyperparameter optimization for multi-disease detection and diagnosis in machine learning
    Singh, Jagandeep
    Sandhu, Jasminder Kaur
    Kumar, Yogesh
    SERVICE ORIENTED COMPUTING AND APPLICATIONS, 2024, 18 (02) : 163 - 182
  • [44] An improved approach to Arabic news classification based on hyperparameter tuning of machine learning algorithms
    Jamaleddyn, Imad
    El Ayachi, Rachid
    Biniz, Mohamed
    JOURNAL OF ENGINEERING RESEARCH, 2023, 11 (02):
  • [45] Grid search in hyperparameter optimization of machine learning models for prediction of HIV/AIDS test results
    Belete D.M.
    Huchaiah M.D.
    International Journal of Computers and Applications, 2022, 44 (09) : 875 - 886
  • [46] A Surrogate Model Assisted Quantum-inspired Evolutionary Algorithm for Hyperparameter Optimization in Machine Learning
    Peng, Cheng
    Li, Yangyang
    Cao, Lei
    Jiao, Licheng
    2019 IEEE CONGRESS ON EVOLUTIONARY COMPUTATION (CEC), 2019, : 1060 - 1067
  • [47] Reinforcement Learning for Model Selection and Hyperparameter Optimization
    Wu J.
    Chen S.-P.
    Chen X.-Y.
    Zhou R.
    Dianzi Keji Daxue Xuebao/Journal of the University of Electronic Science and Technology of China, 2020, 49 (02): : 255 - 261
  • [48] Machine Learning Assisted Evolutionary Multi-Objective Optimization
    Zhang, Xingyi
    Cheng, Ran
    Feng, Liang
    Jin, Yaochu
    IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE, 2023, 18 (02) : 16 - 17
  • [49] Hyperparameter optimization: a comparative machine learning model analysis for enhanced heart disease prediction accuracy
    Rimal, Yagyanath
    Sharma, Navneet
    MULTIMEDIA TOOLS AND APPLICATIONS, 2023, 83 (18) : 55091 - 55107
  • [50] Metaheuristic-based hyperparameter optimization for multi-disease detection and diagnosis in machine learning
    Jagandeep Singh
    Jasminder Kaur Sandhu
    Yogesh Kumar
    Service Oriented Computing and Applications, 2024, 18 : 163 - 182