T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917941196357632 |
|---|---|
| author | Moon, Seunghyun Li, Mao Chen, Gregory Knag, Phil Krishnamurthy, Ram Seok, Mingoo |
| author_facet | Moon, Seunghyun Li, Mao Chen, Gregory Knag, Phil Krishnamurthy, Ram Seok, Mingoo |
| contents | This work introduces novel training and post-training compression schemes to reduce external memory access during transformer model inference. Additionally, a new control flow mechanism, called dynamic batching, and a novel buffer architecture, termed a two-direction accessible register file, further reduce external memory access while improving hardware utilization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_00322 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET Moon, Seunghyun Li, Mao Chen, Gregory Knag, Phil Krishnamurthy, Ram Seok, Mingoo Hardware Architecture Artificial Intelligence This work introduces novel training and post-training compression schemes to reduce external memory access during transformer model inference. Additionally, a new control flow mechanism, called dynamic batching, and a novel buffer architecture, termed a two-direction accessible register file, further reduce external memory access while improving hardware utilization. |
| title | T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET |
| topic | Hardware Architecture Artificial Intelligence |
| url | https://arxiv.org/abs/2503.00322 |