Transient dynamics of associative memory models
Abstract
Associative memory models such as the Hopfield network and its dense generalizations with higher-order interactions exhibit a “blackout catastrophe”–a discontinuous transition where stable memory states abruptly vanish when the number of stored patterns exceeds a critical capacity. This transition is often interpreted as rendering networks unusable beyond capacity limits. We argue that this interpretation is largely an artifact of the equilibrium perspective. We derive dynamical mean-field equations using a bipartite cavity approach for graded-activity dense associative memory models, with the Hopfield model as a special case, and solve them using a numerical scheme. We show that patterns can be transiently retrieved with high accuracy above capacity despite the absence of stable attractors. This occurs because slow regions persist in the above-capacity energy landscape as shallow, unstable remnants of below-capacity stable basins. The same transient-retrieval effect occurs in below-capacity networks initialized outside basins of attraction. “Transient-recovery curves” provide a concise visual summary of these effects, revealing graceful, non-catastrophic changes in retrieval behavior above capacity and allowing us to compare the behavior across interaction orders. This dynamical perspective reveals rich energy landscape structure obscured by equilibrium analysis and suggests biological neural circuits may exploit transient dynamics for memory retrieval. Furthermore, our approach suggests ways of understanding computational properties of neural circuits without reference to fixed points, advances the technical repertoire of numerical mean-field solution methods for recurrent neural networks, and yields new theoretical results on generalizations of the Hopfield model.
