A CONVEXITY-BASED ANALYSIS OF NEURAL NETWORKS

This investigation presents a convexity-based method for determining the class of functions realizable by a neural network. Other techniques for analyzing neural nets require that the set of networks form a linear subspace. The principal advantage of this approach is that it permits the analysis of convex sets of functions, in particular the method applies to networks that use only nonnegative weights. The interest in a network that uses unipolar weights arises from the problem of representing bipolar weights in optical neural nets. A unipolar network that has the desired approximation properties provides a simple solution to this problem. We show that nonnegative linear combinations and compositions of excitatory and inhibitory response functions uniformly approximate arbitrary nonnegative continuous functions.