Towards Active Synthetic Data Generation for Finetuning Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910016126058496 |
|---|---|
| author | Kessler, Samuel Xia, Menglin Diaz, Daniel Madrigal Han, Dongge Heshemi, Helia Rajmohan, Saravan Ruehle, Victor Ash, Jordan T. |
| author_facet | Kessler, Samuel Xia, Menglin Diaz, Daniel Madrigal Han, Dongge Heshemi, Helia Rajmohan, Saravan Ruehle, Victor Ash, Jordan T. |
| contents | A common and effective means for improving language model capabilities involves finetuning a ``student'' language model's parameters on generations from a more proficient ``teacher'' model. Termed ``synthetic data'', these generations are often produced before any student finetuning, but some work has considered generating new synthetic samples as training progresses. This paper studies and advocates for the latter case, where data are generated in an iterative, closed-loop fashion that is guided by the current state of the student model. For a fixed budget of generated samples, or a budget in terms of compute spent querying a teacher, we show that this curation of finetuning data affords improved student performance over static generation. Further, while there have been several LLM-specific methods proposed that operate in this regime, we find that simple, inexpensive selection criteria from the active learning literature tend to be most performant. We validate these claims across four mathematical and logical reasoning datasets using four different small language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_00884 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Active Synthetic Data Generation for Finetuning Language Models Kessler, Samuel Xia, Menglin Diaz, Daniel Madrigal Han, Dongge Heshemi, Helia Rajmohan, Saravan Ruehle, Victor Ash, Jordan T. Machine Learning Computation and Language A common and effective means for improving language model capabilities involves finetuning a ``student'' language model's parameters on generations from a more proficient ``teacher'' model. Termed ``synthetic data'', these generations are often produced before any student finetuning, but some work has considered generating new synthetic samples as training progresses. This paper studies and advocates for the latter case, where data are generated in an iterative, closed-loop fashion that is guided by the current state of the student model. For a fixed budget of generated samples, or a budget in terms of compute spent querying a teacher, we show that this curation of finetuning data affords improved student performance over static generation. Further, while there have been several LLM-specific methods proposed that operate in this regime, we find that simple, inexpensive selection criteria from the active learning literature tend to be most performant. We validate these claims across four mathematical and logical reasoning datasets using four different small language models. |
| title | Towards Active Synthetic Data Generation for Finetuning Language Models |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2512.00884 |