On structural modification learning algorithms

Artificial neural networks, designed to learn from examples rather than explicit programming, have traditionally modified only are weights of the underlying graph structure, leaving the task of choosing an appropriate network graph up to the user During the last decade, a number of researchers have sought to address the problem of modifying the graph as well. In this paper we examine several recent learning algorithms that employ structural modification mechanisms. FUNNET extends backpropagation to include node generation and annihilation. Cascade-Correlation introduces individual training of hidden nodes, taking advantage of all prior nodes. Upstart employs recursion to construct a binary tree network that perfectly classifies hypercubic training sets. Having discussed the strengths and weaknesses of each of these, this paper then presents two hybrid structural modification learning algorithms, each of which draws upon the best features of the three learning algorithms and seeks to eliminate some shortcomings.