Deep Neural Networks as Finite-Step Hopfield Dynamics

Energy-based models describe neural computation as descent on a scalar functional, exemplified by Hopfield networks, whereas modern deep networks are typically viewed as static compositions of nonlinear maps. We show that each layer update can be written as a gradient descent step on a Hopfield-style energy, so network evaluation corresponds to a finite sequence of energy descent steps. We analyze the evolution of small perturbations under this finite-step dynamics without assuming convergence. In a weight-shared setting, perturbation growth is governed by the Jacobian spectral norm, linking stability to weight scale and activation curvature. Motivated by this analysis, we introduce a Jacobian quantile regularizer that suppresses expansive updates during training. Experiments on MNIST show improved robustness to additive perturbations on image while preserving clean accuracy.