A BIOLOGICALLY PLAUSIBLE ASSOCIATIVE MEMORY NETWORK

The Hopfield network has been the leading model for associative memory for over four decades, culminating in the recent 2024 Nobel Prize. However, the vanilla version of the Hopfield network has a capacity that scales with the number of connections per neuron. In the mammalian brain, that’s about 1,000, leading to a capacity of about 50 memories in a spiking network—regardless of its size. Therefore, it cannot possibly account for the capacity of human memory. To address these limitations, various modifications to the Hopfield network have been proposed. One promising variant, Dense Associative Memory or the Modern Hopfield Network, incorporates a two-layer architecture with memory and feature neurons, which significantly increases capacity. However, from the point of view of biological plausibility, this comes with a downside: during recall most neurons in the memory layer are silent, whereas in the brain neurons are rarely silent for extended periods. This is not easy to fix: the memory layer contains a large number of neurons, and allowing silent neurons to exhibit even low firing rates can introduce an unacceptable level of noise, preventing the perfect recall of stored memories. To address this challenge, we introduce nonlinear dendrites in the Dense Associative Memory network. This model supports a capacity that is polynomial in the number of memory neurons while enabling non-recalled memory neurons to exhibit nonzero firing rates. The proposed architecture adheres to key biological constraints, including the presence of both excitatory and inhibitory populations that obey Dale's law and maintain non-saturated firing rates. These properties enhance the model's biological plausibility while achieving polynomial capacity, bridging the gap between theoretical and biological constraints on associative memory.