Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908824270536704 |
|---|---|
| author | Ye, Junjie Yang, Yuming Nan, Yang Li, Shuo Zhang, Qi Gui, Tao Huang, Xuanjing Wang, Peng Shi, Zhongchao Fan, Jianping |
| author_facet | Ye, Junjie Yang, Yuming Nan, Yang Li, Shuo Zhang, Qi Gui, Tao Huang, Xuanjing Wang, Peng Shi, Zhongchao Fan, Jianping |
| contents | Large language models (LLMs) acquire substantial world knowledge during pre-training, which is further shaped by post-training techniques such as supervised fine-tuning (SFT). However, the impact of SFT on a model's knowledge remains underexplored, limiting our ability to control knowledge change behavior in fine-tuned models. To address this gap, we evaluate closed-book question answering (CBQA) performance across five LLMs from the LLaMA-2 and LLaMA-3 families. Surprisingly, models fine-tuned on 1,920 samples perform up to 14% worse than those fine-tuned on only 240 samples. Furthermore, varying the level of knowledge mastery in the fine-tuning data leads to performance fluctuations of over 12%. To investigate these effects, we analyze model behavior at both the token and parameter levels. Our analysis reveals that up to 90% of parameter updates during SFT do not contribute to knowledge enhancement. Restoring these updates can improve performance on the CBQA task, depending on the characteristics of the fine-tuning data. These insights offer practical guidance for developing fine-tuning strategies that more effectively strengthen model knowledge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_16596 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels Ye, Junjie Yang, Yuming Nan, Yang Li, Shuo Zhang, Qi Gui, Tao Huang, Xuanjing Wang, Peng Shi, Zhongchao Fan, Jianping Computation and Language Artificial Intelligence Large language models (LLMs) acquire substantial world knowledge during pre-training, which is further shaped by post-training techniques such as supervised fine-tuning (SFT). However, the impact of SFT on a model's knowledge remains underexplored, limiting our ability to control knowledge change behavior in fine-tuned models. To address this gap, we evaluate closed-book question answering (CBQA) performance across five LLMs from the LLaMA-2 and LLaMA-3 families. Surprisingly, models fine-tuned on 1,920 samples perform up to 14% worse than those fine-tuned on only 240 samples. Furthermore, varying the level of knowledge mastery in the fine-tuning data leads to performance fluctuations of over 12%. To investigate these effects, we analyze model behavior at both the token and parameter levels. Our analysis reveals that up to 90% of parameter updates during SFT do not contribute to knowledge enhancement. Restoring these updates can improve performance on the CBQA task, depending on the characteristics of the fine-tuning data. These insights offer practical guidance for developing fine-tuning strategies that more effectively strengthen model knowledge. |
| title | Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.16596 |