Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917795347824640 |
|---|---|
| author | Ju, Yiming Ni, Ziyi Xing, Xingrun Zeng, Zhixiong Zhao, hanyu Fan, Siqi Zhang, Zheng |
| author_facet | Ju, Yiming Ni, Ziyi Xing, Xingrun Zeng, Zhixiong Zhao, hanyu Fan, Siqi Zhang, Zheng |
| contents | Supervised fine-tuning (SFT) is crucial for adapting Large Language Models (LLMs) to specific tasks. In this work, we demonstrate that the order of training data can lead to significant training imbalances, potentially resulting in performance degradation. Consequently, we propose to mitigate this imbalance by merging SFT models fine-tuned with different data orders, thereby enhancing the overall effectiveness of SFT. Additionally, we introduce a novel technique, "parameter-selection merging," which outperforms traditional weighted-average methods on five datasets. Further, through analysis and ablation studies, we validate the effectiveness of our method and identify the sources of performance improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03743 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging Ju, Yiming Ni, Ziyi Xing, Xingrun Zeng, Zhixiong Zhao, hanyu Fan, Siqi Zhang, Zheng Computation and Language Artificial Intelligence Machine Learning Supervised fine-tuning (SFT) is crucial for adapting Large Language Models (LLMs) to specific tasks. In this work, we demonstrate that the order of training data can lead to significant training imbalances, potentially resulting in performance degradation. Consequently, we propose to mitigate this imbalance by merging SFT models fine-tuned with different data orders, thereby enhancing the overall effectiveness of SFT. Additionally, we introduce a novel technique, "parameter-selection merging," which outperforms traditional weighted-average methods on five datasets. Further, through analysis and ablation studies, we validate the effectiveness of our method and identify the sources of performance improvements. |
| title | Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2410.03743 |