Continual Learning for Object Classification: A Modular Approach

A Human can immediately add new items to its set of known objects, whereas a computer, using traditional computer vision algorithms, would typically have to go almost back to the start and re-learn the all collection of objects (classes) from scratch. The reason the network must be re-trained is due to a phenomenon named Catastrophic Forgetting, where the changes made to the system during the acquisition of new knowledge brings about the loss of previous knowledge. In this paper, we explore the Continual Learning problem by proposing a way to deal with Catastrophic Forgetting. Our proposal includes a framework capable of learning new information without having to start from scratch and even "improve" its knowledge on what it already knows. With the above in mind, we present the Modular Dynamic Neural Network (MDNN), a network primarily made up of modular sub-networks that progressively grows in a tree shape and re-arranges itself as it learns continuously. The network is divided into two main blocks: (a) the feature extraction block, which is based on a ResNet50; and (b) the modular dynamic classification block, which is made up of sub-networks structured in such a way that its internal components function independently from one another. This structure allows that when new information is learned only specific sub-networks are altered in a way that old information is not forgotten. Tests show promising results with a set of ImageNet classes and also with a set of our own classes.