CAMformer: Associative Memory is All You Need
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915636521730048 |
|---|---|
| author | Molom-Ochir, Tergel Morris, Benjamin F. Horton, Mark Wei, Chiyue Guo, Cong Taylor, Brady Liu, Peter Wang, Shan X. Fan, Deliang Li, Hai Helen Chen, Yiran |
| author_facet | Molom-Ochir, Tergel Morris, Benjamin F. Horton, Mark Wei, Chiyue Guo, Cong Taylor, Brady Liu, Peter Wang, Shan X. Fan, Deliang Li, Hai Helen Chen, Yiran |
| contents | Transformers face scalability challenges due to the quadratic cost of attention, which involves dense similarity computations between queries and keys. We propose CAMformer, a novel accelerator that reinterprets attention as an associative memory operation and computes attention scores using a voltage-domain Binary Attention Content Addressable Memory (BA-CAM). This enables constant-time similarity search through analog charge sharing, replacing digital arithmetic with physical similarity sensing. CAMformer integrates hierarchical two-stage top-k filtering, pipelined execution, and high-precision contextualization to achieve both algorithmic accuracy and architectural efficiency. Evaluated on BERT and Vision Transformer workloads, CAMformer achieves over 10x energy efficiency, up to 4x higher throughput, and 6-8x lower area compared to state-of-the-art accelerators--while maintaining near-lossless accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_19740 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CAMformer: Associative Memory is All You Need Molom-Ochir, Tergel Morris, Benjamin F. Horton, Mark Wei, Chiyue Guo, Cong Taylor, Brady Liu, Peter Wang, Shan X. Fan, Deliang Li, Hai Helen Chen, Yiran Hardware Architecture Machine Learning Transformers face scalability challenges due to the quadratic cost of attention, which involves dense similarity computations between queries and keys. We propose CAMformer, a novel accelerator that reinterprets attention as an associative memory operation and computes attention scores using a voltage-domain Binary Attention Content Addressable Memory (BA-CAM). This enables constant-time similarity search through analog charge sharing, replacing digital arithmetic with physical similarity sensing. CAMformer integrates hierarchical two-stage top-k filtering, pipelined execution, and high-precision contextualization to achieve both algorithmic accuracy and architectural efficiency. Evaluated on BERT and Vision Transformer workloads, CAMformer achieves over 10x energy efficiency, up to 4x higher throughput, and 6-8x lower area compared to state-of-the-art accelerators--while maintaining near-lossless accuracy. |
| title | CAMformer: Associative Memory is All You Need |
| topic | Hardware Architecture Machine Learning |
| url | https://arxiv.org/abs/2511.19740 |