CAPACITY OF NEURAL NETWORKS WITH DISCRETE SYNAPTIC COUPLINGS

The authors study the optimal storage capacity of neural networks with discrete local constraints on the synaptic couplings Jij. Models with such constraints include those with binary couplings Jij=+or-1 or Jij=0, 1, quantised couplings with larger synaptic range, e.g. J ij=+or-1/L, +or-2/L, . . ., +or-1 and, in the limit, continuous couplings confined to the hypercube mod Jij mod <or=1 ('box confinement'). They find that the optimal storage capacity alpha ( kappa ) is best determined by the vanishing of a suitably defined 'entropy' as calculated in the replica symmetric approximation. They also extend their results to cases with biased memories and make contact with sparse coding models.