Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.14826 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912385216806912 |
|---|---|
| author | Deb, Rohan Thekumparampil, Kiran Kalantari, Kousha Hiranandani, Gaurush Sabach, Shoham Kveton, Branislav |
| author_facet | Deb, Rohan Thekumparampil, Kiran Kalantari, Kousha Hiranandani, Gaurush Sabach, Shoham Kveton, Branislav |
| contents | Supervised fine-tuning (SFT) is a standard approach to adapting large language models (LLMs) to new domains. In this work, we improve the statistical efficiency of SFT by selecting an informative subset of training examples. Specifically, for a fixed budget of training examples, which determines the computational cost of fine-tuning, we determine the most informative ones. The key idea in our method is to select examples that maximize information gain, measured by the Hessian of the log-likelihood of the LLM. We approximate it efficiently by linearizing the LLM at the last layer using multinomial logistic regression models. Our approach is computationally efficient, analyzable, and performs well empirically. We demonstrate this on several problems, and back our claims with both quantitative results and an LLM evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14826 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain Deb, Rohan Thekumparampil, Kiran Kalantari, Kousha Hiranandani, Gaurush Sabach, Shoham Kveton, Branislav Machine Learning Computation and Language Supervised fine-tuning (SFT) is a standard approach to adapting large language models (LLMs) to new domains. In this work, we improve the statistical efficiency of SFT by selecting an informative subset of training examples. Specifically, for a fixed budget of training examples, which determines the computational cost of fine-tuning, we determine the most informative ones. The key idea in our method is to select examples that maximize information gain, measured by the Hessian of the log-likelihood of the LLM. We approximate it efficiently by linearizing the LLM at the last layer using multinomial logistic regression models. Our approach is computationally efficient, analyzable, and performs well empirically. We demonstrate this on several problems, and back our claims with both quantitative results and an LLM evaluation. |
| title | FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2505.14826 |