STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916076242075648 |
|---|---|
| author | Wu, Dennis Hu, Jerry Yao-Chieh Li, Weijian Chen, Bo-Yu Liu, Han |
| author_facet | Wu, Dennis Hu, Jerry Yao-Chieh Li, Weijian Chen, Bo-Yu Liu, Han |
| contents | We present STanHop-Net (Sparse Tandem Hopfield Network) for multivariate time series prediction with memory-enhanced capabilities. At the heart of our approach is STanHop, a novel Hopfield-based neural network block, which sparsely learns and stores both temporal and cross-series representations in a data-dependent fashion. In essence, STanHop sequentially learn temporal representation and cross-series representation using two tandem sparse Hopfield layers. In addition, StanHop incorporates two additional external memory modules: a Plug-and-Play module and a Tune-and-Play module for train-less and task-aware memory-enhancements, respectively. They allow StanHop-Net to swiftly respond to certain sudden events. Methodologically, we construct the StanHop-Net by stacking STanHop blocks in a hierarchical fashion, enabling multi-resolution feature extraction with resolution-specific sparsity. Theoretically, we introduce a sparse extension of the modern Hopfield model (Generalized Sparse Modern Hopfield Model) and show that it endows a tighter memory retrieval error compared to the dense counterpart without sacrificing memory capacity. Empirically, we validate the efficacy of our framework on both synthetic and real-world settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_17346 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction Wu, Dennis Hu, Jerry Yao-Chieh Li, Weijian Chen, Bo-Yu Liu, Han Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Neural and Evolutionary Computing We present STanHop-Net (Sparse Tandem Hopfield Network) for multivariate time series prediction with memory-enhanced capabilities. At the heart of our approach is STanHop, a novel Hopfield-based neural network block, which sparsely learns and stores both temporal and cross-series representations in a data-dependent fashion. In essence, STanHop sequentially learn temporal representation and cross-series representation using two tandem sparse Hopfield layers. In addition, StanHop incorporates two additional external memory modules: a Plug-and-Play module and a Tune-and-Play module for train-less and task-aware memory-enhancements, respectively. They allow StanHop-Net to swiftly respond to certain sudden events. Methodologically, we construct the StanHop-Net by stacking STanHop blocks in a hierarchical fashion, enabling multi-resolution feature extraction with resolution-specific sparsity. Theoretically, we introduce a sparse extension of the modern Hopfield model (Generalized Sparse Modern Hopfield Model) and show that it endows a tighter memory retrieval error compared to the dense counterpart without sacrificing memory capacity. Empirically, we validate the efficacy of our framework on both synthetic and real-world settings. |
| title | STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2312.17346 |