Leveraging the true depth of LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917186910552064 |
|---|---|
| author | González, Ramón Calvo Paliotta, Daniele Pagliardini, Matteo Jaggi, Martin Fleuret, François |
| author_facet | González, Ramón Calvo Paliotta, Daniele Pagliardini, Matteo Jaggi, Martin Fleuret, François |
| contents | The remarkable capabilities of Large Language Models (LLMs) are overshadowed by their immense computational cost. While recent work has shown that many LLM layers can be reordered or even removed with minimal impact on accuracy, these insights have not been translated into significant inference speedups. To bridge this gap, we introduce a novel method that restructures the computational graph by grouping and evaluating consecutive layer pairs in parallel. This approach, requiring no retraining, yields a 1.19x throughput gain on Llama 2 7B while reducing the average benchmark accuracy by only 1.5\%. We demonstrate the practical value of this method for large-scale LLM deployment and show that some of the lost accuracy can be recovered with lightweight fine-tuning of the parallelized layers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_02790 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Leveraging the true depth of LLMs González, Ramón Calvo Paliotta, Daniele Pagliardini, Matteo Jaggi, Martin Fleuret, François Machine Learning Computation and Language The remarkable capabilities of Large Language Models (LLMs) are overshadowed by their immense computational cost. While recent work has shown that many LLM layers can be reordered or even removed with minimal impact on accuracy, these insights have not been translated into significant inference speedups. To bridge this gap, we introduce a novel method that restructures the computational graph by grouping and evaluating consecutive layer pairs in parallel. This approach, requiring no retraining, yields a 1.19x throughput gain on Llama 2 7B while reducing the average benchmark accuracy by only 1.5\%. We demonstrate the practical value of this method for large-scale LLM deployment and show that some of the lost accuracy can be recovered with lightweight fine-tuning of the parallelized layers. |
| title | Leveraging the true depth of LLMs |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2502.02790 |