Class Incremental Learning: A Review and Performance Evaluation

Machine learning has been successfully applied in many fields such as computer vision, natural language processing, and speech recognition. However, in the current machine learning systems, models are often fixed after training. Consequently, they can only generalize to classes that appear in the training set, and cannot learn newly emerged classes continuously. In real-world applications, new classes or tasks will appear continuously, which requires the model to continuously learn new knowledge without forgetting the knowledge of previous seen classes. The emerging research direction of class incremental learning aims to enable models to continuously learn new classes while preserving the discrimination ability of old classes (defying “catastrophic forgetting”) in the open and dynamic environment. This paper provides a comprehensive overview of class incremental learning (CIL) developed in recent years. Specifically, existing methods are grouped into five categories: parameter regularization based, knowledge distillation based, data replay based, feature replay based and network structure based methods. The advantages and disadvantages of each method are summarized. In addition, extensive experiments are conducted to evaluate and compare those representative methods on benchmark datasets. Finally, this paper prospects the future research directions of class incremental learning. © 2023 Science Press. All rights reserved.