NEURAL NETWORK WITH HIERARCHICAL-CLUSTERING NEAR SATURATION

We perform a detailed investigation of the storage properties of a model for neural networks that exhibits the same organization into clusters as Dyson's hierarchical model for ferromagnetism, combined with Hebb's learning algorithm for an extensive number of stored patterns p = alpha-N, where N is the size of the network. In a previous publication we presented results for the retrieval properties of the model in the case of finite p, showing that together with the original stored pattern or 'ancestor' the system also retrieves a hierarchy of 'descendants'. Here we first perform a signal-to-noise analysis, obtaining a succession of critical storage capacities for the 'ancestor' and its 'descendants' that are below the Hopfield value. Afterwards we apply the statistical mechanics formulation of Amit, Gutfreund and Sompolinsky, to obtain also in this case a succession of critical storage capacities that are below the corresponding value for Hopfield's model. In both cases we consider the ratio of the critical storage capacity for the 'ancestor' to the same quantity as evaluated in Hopfield's model, and we prove rigorously that the signal-to-noise method provides a lower bound for this ratio, that is bounded from above by unity. We present the phase diagram in the alpha-T plane for the particular case of two clusters and one descendant. We observe the existence of two lines T(c)1(alpha) less-than-or-equal-to T(M)1(alpha) such that at T = T(M)1(alpha) the 'ancestor' orders continuously but for T(c)1(alpha) < T < T(M)1(alpha) the global minimum is still given by the spin-glass phase, while for T < T(c)1(alpha) the free energy of the retrieved ancestor becomes a global minimum, just as in Hopfield's model. A new feature of the model studied here is the existence of a third line T(M)2(alpha) < T(c)1(alpha) such that at T = T(M)2(alpha) the 'descendant' orders discontinuously. The existence of a fourth line T(c)2(alpha) < T(M)2(alpha) depends on the strength of the interaction.