Slamming: Training a Speech Language Model on One GPU in a Day
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910962509938688 |
|---|---|
| author | Maimon, Gallil Elmakies, Avishai Adi, Yossi |
| author_facet | Maimon, Gallil Elmakies, Avishai Adi, Yossi |
| contents | We introduce Slam, a recipe for training high-quality Speech Language Models (SLMs) on a single academic GPU in 24 hours. We do so through empirical analysis of model initialisation and architecture, synthetic training data, preference optimisation with synthetic data and tweaking all other components. We empirically demonstrate that this training recipe also scales well with more compute getting results on par with leading SLMs in a fraction of the compute cost. We hope these insights will make SLM training and research more accessible. In the context of SLM scaling laws, our results far outperform predicted compute optimal performance, giving an optimistic view to SLM feasibility. See code, data, models, samples at - https://pages.cs.huji.ac.il/adiyoss-lab/slamming . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15814 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Slamming: Training a Speech Language Model on One GPU in a Day Maimon, Gallil Elmakies, Avishai Adi, Yossi Machine Learning Artificial Intelligence Computation and Language Sound Audio and Speech Processing We introduce Slam, a recipe for training high-quality Speech Language Models (SLMs) on a single academic GPU in 24 hours. We do so through empirical analysis of model initialisation and architecture, synthetic training data, preference optimisation with synthetic data and tweaking all other components. We empirically demonstrate that this training recipe also scales well with more compute getting results on par with leading SLMs in a fraction of the compute cost. We hope these insights will make SLM training and research more accessible. In the context of SLM scaling laws, our results far outperform predicted compute optimal performance, giving an optimistic view to SLM feasibility. See code, data, models, samples at - https://pages.cs.huji.ac.il/adiyoss-lab/slamming . |
| title | Slamming: Training a Speech Language Model on One GPU in a Day |
| topic | Machine Learning Artificial Intelligence Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2502.15814 |