A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912084455849984 |
|---|---|
| author | Rawat, Ankit Singh Sadhanala, Veeranjaneyulu Rostamizadeh, Afshin Chakrabarti, Ayan Jitkrittum, Wittawat Feinberg, Vladimir Kim, Seungyeon Harutyunyan, Hrayr Saunshi, Nikunj Nado, Zachary Shivanna, Rakesh Reddi, Sashank J. Menon, Aditya Krishna Anil, Rohan Kumar, Sanjiv |
| author_facet | Rawat, Ankit Singh Sadhanala, Veeranjaneyulu Rostamizadeh, Afshin Chakrabarti, Ayan Jitkrittum, Wittawat Feinberg, Vladimir Kim, Seungyeon Harutyunyan, Hrayr Saunshi, Nikunj Nado, Zachary Shivanna, Rakesh Reddi, Sashank J. Menon, Aditya Krishna Anil, Rohan Kumar, Sanjiv |
| contents | A primary challenge in large language model (LLM) development is their onerous pre-training cost. Typically, such pre-training involves optimizing a self-supervised objective (such as next-token prediction) over a large corpus. This paper explores a promising paradigm to improve LLM pre-training efficiency and quality by suitably leveraging a small language model (SLM). In particular, this paradigm relies on an SLM to both (1) provide soft labels as additional training supervision, and (2) select a small subset of valuable ("informative" and "hard") training examples. Put together, this enables an effective transfer of the SLM's predictive distribution to the LLM, while prioritizing specific regions of the training data distribution. Empirically, this leads to reduced LLM training time compared to standard training, while improving the overall quality. Theoretically, we develop a statistical framework to systematically study the utility of SLMs in enabling efficient training of high-quality LLMs. In particular, our framework characterizes how the SLM's seemingly low-quality supervision can enhance the training of a much more capable LLM. Furthermore, it also highlights the need for an adaptive utilization of such supervision, by striking a balance between the bias and variance introduced by the SLM-provided soft labels. We corroborate our theoretical framework by improving the pre-training of an LLM with 2.8B parameters by utilizing a smaller LM with 1.5B parameters on the Pile dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_18779 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs Rawat, Ankit Singh Sadhanala, Veeranjaneyulu Rostamizadeh, Afshin Chakrabarti, Ayan Jitkrittum, Wittawat Feinberg, Vladimir Kim, Seungyeon Harutyunyan, Hrayr Saunshi, Nikunj Nado, Zachary Shivanna, Rakesh Reddi, Sashank J. Menon, Aditya Krishna Anil, Rohan Kumar, Sanjiv Machine Learning Computation and Language A primary challenge in large language model (LLM) development is their onerous pre-training cost. Typically, such pre-training involves optimizing a self-supervised objective (such as next-token prediction) over a large corpus. This paper explores a promising paradigm to improve LLM pre-training efficiency and quality by suitably leveraging a small language model (SLM). In particular, this paradigm relies on an SLM to both (1) provide soft labels as additional training supervision, and (2) select a small subset of valuable ("informative" and "hard") training examples. Put together, this enables an effective transfer of the SLM's predictive distribution to the LLM, while prioritizing specific regions of the training data distribution. Empirically, this leads to reduced LLM training time compared to standard training, while improving the overall quality. Theoretically, we develop a statistical framework to systematically study the utility of SLMs in enabling efficient training of high-quality LLMs. In particular, our framework characterizes how the SLM's seemingly low-quality supervision can enhance the training of a much more capable LLM. Furthermore, it also highlights the need for an adaptive utilization of such supervision, by striking a balance between the bias and variance introduced by the SLM-provided soft labels. We corroborate our theoretical framework by improving the pre-training of an LLM with 2.8B parameters by utilizing a smaller LM with 1.5B parameters on the Pile dataset. |
| title | A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2410.18779 |