Addressing Catastrophic Forgetting in Federated Learning on Resource-Constrained Devices: A Feature Replay Approach

Federated learning is a new type of privacy-preserving distributed machine learning method. Despite its advantages, Federated learning suffers from catastrophic forgetting due to the temporal variability of data on participating devices. While some methods have been proposed to address this issue in federated learning, they often come at the expense of storage space, which is not a practical solution. This paper presents a federated incremental learning framework designed specifically for resource-constrained devices to address catastrophic forgetting. Leveraging a feature generator, knowledge distillation, and dynamic adaptive weight allocation, the framework addresses catastrophic forgetting and accelerates model convergence. Our framework effectively addresses the issue of catastrophic forgetting, even in the context of resource-constrained devices with limited storage. The results demonstrate significant improvements in our framework compared to existing baselines on the CIFAR-10 and CIFAR-100 datasets.