SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908795063500800 |
|---|---|
| author | Lin, Jiacheng Wang, Zhongruo Qian, Kun Wang, Tian Srinivasan, Arvind Zeng, Hansi Jiao, Ruochen Zhou, Xie Gesi, Jiri Wang, Dakuo Guo, Yufan Zhong, Kai Zhang, Weiqi Sanghavi, Sujay Chen, Changyou Yun, Hyokun Li, Lihong |
| author_facet | Lin, Jiacheng Wang, Zhongruo Qian, Kun Wang, Tian Srinivasan, Arvind Zeng, Hansi Jiao, Ruochen Zhou, Xie Gesi, Jiri Wang, Dakuo Guo, Yufan Zhong, Kai Zhang, Weiqi Sanghavi, Sujay Chen, Changyou Yun, Hyokun Li, Lihong |
| contents | Supervised Fine-Tuning (SFT) on domain-specific datasets is a common approach to adapt Large Language Models (LLMs) to specialized tasks but is often believed to degrade their general capabilities. In this work, we revisit this trade-off and present both empirical and theoretical insights. First, we show that SFT does not always hurt: using a smaller learning rate can substantially mitigate general performance degradation while preserving comparable target-domain performance. We then provide a theoretical analysis that explains these phenomena and further motivates a new method, Token-Adaptive Loss Reweighting (TALR). Building on this, and recognizing that smaller learning rates alone do not fully eliminate general-performance degradation in all cases, we evaluate a range of strategies for reducing general capability loss, including L2 regularization, LoRA, model averaging, FLOW, and our proposed TALR. Experimental results demonstrate that while no method completely eliminates the trade-off, TALR consistently outperforms these baselines in balancing domain-specific gains and general capabilities. Finally, we distill our findings into practical guidelines for adapting LLMs to new domains: (i) using a small learning rate to achieve a favorable trade-off, and (ii) when a stronger balance is further desired, adopt TALR as an effective strategy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20758 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs Lin, Jiacheng Wang, Zhongruo Qian, Kun Wang, Tian Srinivasan, Arvind Zeng, Hansi Jiao, Ruochen Zhou, Xie Gesi, Jiri Wang, Dakuo Guo, Yufan Zhong, Kai Zhang, Weiqi Sanghavi, Sujay Chen, Changyou Yun, Hyokun Li, Lihong Computation and Language Supervised Fine-Tuning (SFT) on domain-specific datasets is a common approach to adapt Large Language Models (LLMs) to specialized tasks but is often believed to degrade their general capabilities. In this work, we revisit this trade-off and present both empirical and theoretical insights. First, we show that SFT does not always hurt: using a smaller learning rate can substantially mitigate general performance degradation while preserving comparable target-domain performance. We then provide a theoretical analysis that explains these phenomena and further motivates a new method, Token-Adaptive Loss Reweighting (TALR). Building on this, and recognizing that smaller learning rates alone do not fully eliminate general-performance degradation in all cases, we evaluate a range of strategies for reducing general capability loss, including L2 regularization, LoRA, model averaging, FLOW, and our proposed TALR. Experimental results demonstrate that while no method completely eliminates the trade-off, TALR consistently outperforms these baselines in balancing domain-specific gains and general capabilities. Finally, we distill our findings into practical guidelines for adapting LLMs to new domains: (i) using a small learning rate to achieve a favorable trade-off, and (ii) when a stronger balance is further desired, adopt TALR as an effective strategy. |
| title | SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.20758 |