DOSI: Training Artificial Neural Networks using Overlapping Swarm Intelligence with Local Credit Assignment

A novel swarm-based algorithm is proposed for the training of artificial neural networks. Training of such networks is a difficult problem that requires an effective search algorithm to find optimal weight values. While gradient-based methods, such as backpropagation, are frequently used to train multi-layer feedforward neural networks, such methods may not yield a globally optimal solution. To overcome the limitations of gradient-based methods, evolutionary algorithms have been used to train these networks with some success. This paper proposes an overlapping swarm intelligence algorithm for training neural networks in which a particle swarm is assigned to each neuron to search for that neuron's weights. Unlike similar architectures, our approach does not require a shared global network for fitness evaluation. Thus the approach discussed in this paper localizes the credit assignment process by first focusing on updating weights within local swarms and then evaluating the fitness of the particles using a localized network. This has the advantage of enabling our algorithm's learning process to be fully distributed.