STATISTICAL-MECHANICS OF TRAINING IN NEURAL NETWORKS

A statistical mechanical approach for neural networks in which couplings are the slow dynamical variables is considered. The couplings are assumed to be confined in a restricted subspace near the Hebb-rule structure corresponding to quenched patterns. We study the situation when the couplings thermalize at a temperature different from that of the spin degrees of freedom, which makes it possible to treat the system in terms of the traditional replica approach with a finite number of replicas. The structure of the model is such that the effective evolution of the couplings tends to deepen the free energy minima corresponding to the learnt patterns. The phase diagram obtained exhibits a substantial increase of the retrieval region.