Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.03678 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909825256914944 |
|---|---|
| author | Song, Zhao Xie, Shenghao Zhou, Samson |
| author_facet | Song, Zhao Xie, Shenghao Zhou, Samson |
| contents | This paper studies the computational challenges of large-scale attention-based models in artificial intelligence by utilizing importance sampling methods in the streaming setting. Inspired by the classical definition of the $\ell_2$ sampler and the recent progress of the attention scheme in Large Language Models (LLMs), we propose the definition of the attention sampler. Our approach significantly reduces the computational burden of traditional attention mechanisms. We analyze the effectiveness of the attention sampler from a theoretical perspective, including space and update time. Additionally, our framework exhibits scalability and broad applicability across various model architectures and domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03678 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Sampling Data Structures for Tensor Products in Turnstile Streams Song, Zhao Xie, Shenghao Zhou, Samson Machine Learning This paper studies the computational challenges of large-scale attention-based models in artificial intelligence by utilizing importance sampling methods in the streaming setting. Inspired by the classical definition of the $\ell_2$ sampler and the recent progress of the attention scheme in Large Language Models (LLMs), we propose the definition of the attention sampler. Our approach significantly reduces the computational burden of traditional attention mechanisms. We analyze the effectiveness of the attention sampler from a theoretical perspective, including space and update time. Additionally, our framework exhibits scalability and broad applicability across various model architectures and domains. |
| title | Towards Sampling Data Structures for Tensor Products in Turnstile Streams |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.03678 |