Interactive Effect of Learning Rate and Batch Size to Implement Transfer Learning for Brain Tumor Classification

被引:13
作者
Usmani, Irfan Ahmed [1 ]
Qadri, Muhammad Tahir [1 ]
Zia, Razia [1 ]
Alrayes, Fatma S. S. [2 ]
Saidani, Oumaima [2 ]
Dashtipour, Kia [3 ]
机构
[1] Sir Syed Univ Engn & Technol, Fac Elect & Comp Engn, Karachi 75300, Pakistan
[2] Princess Nourah Bint Abdulrahman Univ, Coll Comp & Informat Sci, Dept Informat Syst, POB 84428, Riyadh 11671, Saudi Arabia
[3] Edinburgh Napier Univ, Sch Comp, Edinburgh EH11 4BN, Scotland
关键词
brain tumor classification; transfer learning; learning rate; batch size; ANOVA analysis; hyperparameter;
D O I
10.3390/electronics12040964
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
For classifying brain tumors with small datasets, the knowledge-based transfer learning (KBTL) approach has performed very well in attaining an optimized classification model. However, its successful implementation is typically affected by different hyperparameters, specifically the learning rate (LR), batch size (BS), and their joint influence. In general, most of the existing research could not achieve the desired performance because the work addressed only one hyperparameter tuning. This study adopted a Cartesian product matrix-based approach, to interpret the effect of both hyperparameters and their interaction on the performance of models. To evaluate their impact, 56 two-tuple hyperparameters from the Cartesian product matrix were used as inputs to perform an extensive exercise, comprising 504 simulations for three cutting-edge architecture-based pre-trained Deep Learning (DL) models, ResNet18, ResNet50, and ResNet101. Additionally, the impact was also assessed by using three well-known optimizers (solvers): SGDM, Adam, and RMSProp. The performance assessment showed that the framework is an efficient framework to attain optimal values of two important hyperparameters (LR and BS) and consequently an optimized model with an accuracy of 99.56%. Further, our results showed that both hyperparameters have a significant impact individually as well as interactively, with a trade-off in between. Further, the evaluation space was extended by using the statistical ANOVA analysis to validate the main findings. F-test returned with p < 0.05, confirming that both hyperparameters not only have a significant impact on the model performance independently, but that there exists an interaction between the hyperparameters for a combination of their levels.
引用
收藏
页数:23
相关论文
共 40 条
  • [1] Afshar P, 2018, IEEE IMAGE PROC, P3129, DOI 10.1109/ICIP.2018.8451379
  • [2] Bahmani M., 2021, arXiv
  • [3] Bengio Yoshua, 2012, Neural Networks: Tricks of the Trade. Second Edition: LNCS 7700, P437, DOI 10.1007/978-3-642-35289-8_26
  • [4] Brain Tumor Imaging
    Brindle, Kevin M.
    Izquierdo-Garcia, Jose L.
    Lewis, David Y.
    Mair, Richard J.
    Wright, Alan J.
    [J]. JOURNAL OF CLINICAL ONCOLOGY, 2017, 35 (21) : 2432 - +
  • [5] Cheng Jun, 2017, Figshare, DOI 10.6084/M9.FIGSHARE.1512427.V5
  • [6] Retrieval of Brain Tumors by Adaptive Spatial Pooling and Fisher Vector Representation
    Cheng, Jun
    Yang, Wei
    Huang, Meiyan
    Huang, Wei
    Jiang, Jun
    Zhou, Yujia
    Yang, Ru
    Zhao, Jie
    Feng, Yanqiu
    Feng, Qianjin
    Chen, Wufan
    [J]. PLOS ONE, 2016, 11 (06):
  • [7] Enhanced Performance of Brain Tumor Classification via Tumor Region Augmentation and Partition
    Cheng, Jun
    Huang, Wei
    Cao, Shuangliang
    Yang, Ru
    Yang, Wei
    Yun, Zhaoqiang
    Wang, Zhijian
    Feng, Qianjin
    [J]. PLOS ONE, 2015, 10 (10):
  • [8] Brain tumor classification using deep CNN features via transfer learning
    Deepak, S.
    Ameer, P. M.
    [J]. COMPUTERS IN BIOLOGY AND MEDICINE, 2019, 111
  • [9] Drevelegas A, 2011, IMAGING OF BRAIN TUMORS WITH HISTOLOGICAL CORRELATIONS, SECOND EDITION, P13, DOI 10.1007/978-3-540-87650-2_2
  • [10] Deep Residual Learning for Image Recognition
    He, Kaiming
    Zhang, Xiangyu
    Ren, Shaoqing
    Sun, Jian
    [J]. 2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, : 770 - 778