Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2503.00322 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917941196357632 |
|---|---|
| author | Moon, Seunghyun Li, Mao Chen, Gregory Knag, Phil Krishnamurthy, Ram Seok, Mingoo |
| author_facet | Moon, Seunghyun Li, Mao Chen, Gregory Knag, Phil Krishnamurthy, Ram Seok, Mingoo |
| contents | This work introduces novel training and post-training compression schemes to reduce external memory access during transformer model inference. Additionally, a new control flow mechanism, called dynamic batching, and a novel buffer architecture, termed a two-direction accessible register file, further reduce external memory access while improving hardware utilization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_00322 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET Moon, Seunghyun Li, Mao Chen, Gregory Knag, Phil Krishnamurthy, Ram Seok, Mingoo Hardware Architecture Artificial Intelligence This work introduces novel training and post-training compression schemes to reduce external memory access during transformer model inference. Additionally, a new control flow mechanism, called dynamic batching, and a novel buffer architecture, termed a two-direction accessible register file, further reduce external memory access while improving hardware utilization. |
| title | T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET |
| topic | Hardware Architecture Artificial Intelligence |
| url | https://arxiv.org/abs/2503.00322 |