Resource-Efficient Language Models: Quantization for Fast and Accessible Inference
Fuente:
arXiv
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918019111845888 |
|---|---|
| author | Jørgensen, Tollef Emil |
| author_facet | Jørgensen, Tollef Emil |
| contents | Large language models have significantly advanced natural language processing, yet their heavy resource demands pose severe challenges regarding hardware accessibility and energy consumption. This paper presents a focused and high-level review of post-training quantization (PTQ) techniques designed to optimize the inference efficiency of LLMs by the end-user, including details on various quantization schemes, granularities, and trade-offs. The aim is to provide a balanced overview between the theory and applications of post-training quantization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_08620 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Resource-Efficient Language Models: Quantization for Fast and Accessible Inference Jørgensen, Tollef Emil Artificial Intelligence 68T07 I.2.0 Large language models have significantly advanced natural language processing, yet their heavy resource demands pose severe challenges regarding hardware accessibility and energy consumption. This paper presents a focused and high-level review of post-training quantization (PTQ) techniques designed to optimize the inference efficiency of LLMs by the end-user, including details on various quantization schemes, granularities, and trade-offs. The aim is to provide a balanced overview between the theory and applications of post-training quantization. |
| title | Resource-Efficient Language Models: Quantization for Fast and Accessible Inference |
| topic | Artificial Intelligence 68T07 I.2.0 |
| url | https://arxiv.org/abs/2505.08620 |