Investigating Catastrophic Forgetting of Deep Learning Models Within Office 31 Dataset

被引:0
作者
Hidayaturrahman [1 ]
Trisetyarso, Agung [2 ]
Kartowisastro, Iman Herwidiana [1 ,3 ]
Budiharto, Widodo [4 ]
机构
[1] Bina Nusantara Univ, Comp Sci Dept, BINUS Grad Program Doctor Comp Sci, Jakarta 11480, Indonesia
[2] Bina Nusantara Univ, Sch Comp Sci, Math Dept, Jakarta 11480, Indonesia
[3] Bina Nusantara Univ, Fac Engn, Comp Engn Dept, Jakarta 11480, Indonesia
[4] Bina Nusantara Univ, Sch Comp Sci, Comp Sci Dept, Jakarta 11480, Indonesia
来源
IEEE ACCESS | 2024年 / 12卷
关键词
Catastrophic forgetting; deep learning; office31; dataset; domain adaptation; RECOGNITION;
D O I
10.1109/ACCESS.2024.3465491
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Deep learning models have shown impressive performance in various tasks. However, they are prone to a phenomenon called catastrophic forgetting. This means they do not remember what they have learned when training on new tasks. In this research paper, we focus on catastrophic forgetting within the context of the Office 31 dataset. We employ five popular deep learning models: EfficientNet, Inception, MobileNet, ResNet, and Vision Transformer. By training and fine-tuning these models on different combinations of domains within the dataset, we analyze their resistance to catastrophic forgetting. Our research findings reveal significant variations in their performance across models and domains. We found that Vision Transformer has a remarkable resilience to catastrophic forgetting, indicating potential domain similarities. In contrast, Inception is the most effective model for generalizing to the target domain before being finetuned using the target dataset. Furthermore, we observed anomalies in the Office 31 dataset as a benchmark for catastrophic forgetting. Therefore, we employed a different data usage strategy to evaluate the occurrence of catastrophic forgetting. This approach amplifies the prior findings already demonstrated by the original dataset.
引用
收藏
页码:138501 / 138509
页数:9
相关论文
共 35 条
[1]   Pseudo-rehearsal: Achieving deep reinforcement learning without catastrophic forgetting [J].
Atkinson, Craig ;
McCane, Brendan ;
Szymanski, Lech ;
Robins, Anthony .
NEUROCOMPUTING, 2021, 428 :291-307
[2]   Unsupervised Domain Adaptation for Grade Prediction of Froth Flotation Based on Wasserstein Distance and Transformer [J].
Cen, Lihui ;
Li, Xuanpu ;
Chen, Xiaofang ;
Xie, Yongfang ;
Tang, Zhaohui .
JOM, 2024, 76 (5) :2362-2371
[3]  
Dosovitskiy A, 2021, INT C LEARN REPR
[4]  
[范苍宁 Fan Cangning], 2021, [自动化学报, Acta Automatica Sinica], V47, P515
[5]   Deep learning-based image recognition for autonomous driving [J].
Fujiyoshi, Hironobu ;
Hirakawa, Tsubasa ;
Yamashita, Takayoshi .
IATSS RESEARCH, 2019, 43 (04) :244-252
[6]  
Ganin Y, 2016, J MACH LEARN RES, V17
[7]   Semi-Supervised Cloud Detection in Satellite Images by Considering the Domain Shift Problem [J].
Guo, Jianhua ;
Xu, Qingsong ;
Zeng, Yue ;
Liu, Zhiheng ;
Zhu, Xiaoxiang .
REMOTE SENSING, 2022, 14 (11)
[8]   A Survey on Vision Transformer [J].
Han, Kai ;
Wang, Yunhe ;
Chen, Hanting ;
Chen, Xinghao ;
Guo, Jianyuan ;
Liu, Zhenhua ;
Tang, Yehui ;
Xiao, An ;
Xu, Chunjing ;
Xu, Yixing ;
Yang, Zhaohui ;
Zhang, Yiman ;
Tao, Dacheng .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2023, 45 (01) :87-110
[9]  
Harjoseputro Y., 2020, International Journal on Advanced Science, Engineering and Information Technology, V10, P2290, DOI DOI 10.18517/IJASEIT.10.6.10948
[10]  
He H., 2023, P INT C MACH LEARN, V202, P12746