On low-dimensional representations and associative memory energy functions

Associative memory (AM) models encode memories as local minima of energy functions and are generally analyzed in terms of their storage capacity. We provide a complementary perspective: minimizing the AM energy function is related to minimizing the dimensionality of the model’s representation. Starting from a representational objective that balances input reconstruction with maximization of the inverse participation ratio (IPR), we relate the objective to minimization of the classical AM energy function. We further generalize the relation to dense AM energy functions using kernel methods.