Skip to yearly menu bar Skip to main content


STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction

Yu-Hsuan Wu · Jerry Hu · Weijian Li · Bo-Yu Chen · Han Liu

Halle B #271
[ ]
Tue 7 May 1:45 a.m. PDT — 3:45 a.m. PDT


We present STanHop-Net (Sparse Tandem Hopfield Network) for multivariate time series prediction with memory-enhanced capabilities. At the heart of our approach is STanHop, a novel Hopfield-based neural network block, which sparsely learns and stores both temporal and cross-series representations in a data-dependent fashion. In essence, STanHop sequentially learns temporal representation and cross-series representation using two tandem sparse Hopfield layers. Additionally, STanHop incorporates two external memory modules: Plug-and-Play and Tune-and-Play for train-less and task-aware memory enhancements, respectively. They allow StanHop-Net to swiftly respond to sudden events. Methodologically, we construct the STanHop-Net by stacking STanHop blocks in a hierarchical fashion, enabling multi-resolution feature extraction with resolution-specific sparsity. Theoretically, we introduce a unified construction (Generalized Sparse Modern Hopfield Model) for both dense and sparse modern Hopfield models and show that it endows a tighter memory retrieval error compared to the dense counterpart without sacrificing memory capacity. Empirically, we validate the efficacy of STanHop-Net on many settings: time series prediction, fast test-time adaptation, and strongly correlated time series prediction.

Chat is not available.