Improving Instruct Models for Free: A Study on Partial Adaptation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909799217627136 |
|---|---|
| author | İrsoy, Ozan Cheng, Pengxiang Chen, Jennifer L. Preoţiuc-Pietro, Daniel Zhang, Shiyue Pappadopulo, Duccio |
| author_facet | İrsoy, Ozan Cheng, Pengxiang Chen, Jennifer L. Preoţiuc-Pietro, Daniel Zhang, Shiyue Pappadopulo, Duccio |
| contents | Instruct models, obtained from various instruction tuning or post-training steps, are commonly deemed superior and more usable than their base counterpart. While the model gains instruction following ability, instruction tuning may lead to forgetting the knowledge from pre-training or it may encourage the model to become overly conversational or verbose. This, in turn, can lead to degradation of in-context few-shot learning performance. In this work, we study the performance trajectory between base and instruct models by scaling down the strength of instruction-tuning via the partial adaption method. We show that, across several model families and model sizes, reducing the strength of instruction-tuning results in material improvement on a few-shot in-context learning benchmark covering a variety of classic natural language tasks. This comes at the cost of losing some degree of instruction following ability as measured by AlpacaEval. Our study shines light on the potential trade-off between in-context learning and instruction following abilities that is worth considering in practice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_11626 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving Instruct Models for Free: A Study on Partial Adaptation İrsoy, Ozan Cheng, Pengxiang Chen, Jennifer L. Preoţiuc-Pietro, Daniel Zhang, Shiyue Pappadopulo, Duccio Computation and Language Artificial Intelligence Instruct models, obtained from various instruction tuning or post-training steps, are commonly deemed superior and more usable than their base counterpart. While the model gains instruction following ability, instruction tuning may lead to forgetting the knowledge from pre-training or it may encourage the model to become overly conversational or verbose. This, in turn, can lead to degradation of in-context few-shot learning performance. In this work, we study the performance trajectory between base and instruct models by scaling down the strength of instruction-tuning via the partial adaption method. We show that, across several model families and model sizes, reducing the strength of instruction-tuning results in material improvement on a few-shot in-context learning benchmark covering a variety of classic natural language tasks. This comes at the cost of losing some degree of instruction following ability as measured by AlpacaEval. Our study shines light on the potential trade-off between in-context learning and instruction following abilities that is worth considering in practice. |
| title | Improving Instruct Models for Free: A Study on Partial Adaptation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.11626 |