Non-negative Pyramidal Neural Network for Parts-based Learning

Lateral Inhibition Pyramidal Neural Network (LIP-Net) is a type of pyramidal neural network efficiently used in computer vision for pattern recognition. The first set of layers of the network is responsible for the implicit feature extraction; the second and final set of layers, then, classifies the input patterns. This network incorporates in its architecture some biological inspirations such as the deep-learning held in the brain and the existence of receptive and inhibitory fields in the human visual system. Additionally, there is another theory based on the functioning of biological neural networks not considered in the LIPNet: parts-based learning. This type of learning aims to understand a pattern starting with its simplest concepts. Studies have shown that one can achieve this learning by introducing a non-negative constraint on the LIPNet weights. This way, the non-negative network could get an interpretable and not opaque learning distinguishing it from traditional neural networks. Thus, we use Particle Swarm Optimization (PSO) in this paper to apply this constraint on the LIPNet. We also propose a display model of the network learning. Through this model, it is possible to compare the internal representation of the non-negative model learning and the traditional LIPNet model. The results show that the LIPNet with non-negative constraint presented a better interpretability of the patterns.