A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913513491922944 |
|---|---|
| author | Meyer, Svea Marie Weidel, Philipp Plank, Philipp Campos-Macias, Leobardo Shrestha, Sumit Bam Stratmann, Philipp Richter, Mathis |
| author_facet | Meyer, Svea Marie Weidel, Philipp Plank, Philipp Campos-Macias, Leobardo Shrestha, Sumit Bam Stratmann, Philipp Richter, Mathis |
| contents | Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15022 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing Meyer, Svea Marie Weidel, Philipp Plank, Philipp Campos-Macias, Leobardo Shrestha, Sumit Bam Stratmann, Philipp Richter, Mathis Machine Learning Artificial Intelligence Emerging Technologies Neural and Evolutionary Computing Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs. |
| title | A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing |
| topic | Machine Learning Artificial Intelligence Emerging Technologies Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2409.15022 |