If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910811641872384 |
|---|---|
| author | Khalifa, Muhammad Tan, Yi-Chern Ahmadian, Arash Hosking, Tom Lee, Honglak Wang, Lu Üstün, Ahmet Sherborne, Tom Gallé, Matthias |
| author_facet | Khalifa, Muhammad Tan, Yi-Chern Ahmadian, Arash Hosking, Tom Lee, Honglak Wang, Lu Üstün, Ahmet Sherborne, Tom Gallé, Matthias |
| contents | Model merging has shown great promise at combining expert models, but the benefit of merging is unclear when merging "generalist" models trained on many tasks. We explore merging in the context of large (~100B) models, by recycling checkpoints that exhibit tradeoffs among different tasks. Such checkpoints are often created in the process of developing a frontier model, and the suboptimal ones are usually discarded. Given a pool of model checkpoints obtained from different training runs (e.g., different stages, objectives, hyperparameters, and data mixtures), which naturally show tradeoffs across different language capabilities (e.g., instruction following vs. code generation), we investigate whether merging can recycle such suboptimal models into a Pareto-optimal one. Our optimization algorithm tunes the weight of each checkpoint in a linear combination, resulting in such an optimal model that outperforms both individual models and merge-based baselines. Further analysis shows that good merges tend to include almost all checkpoints with non-zero weights, indicating that even seemingly bad initial checkpoints can contribute to good final merges. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04144 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs Khalifa, Muhammad Tan, Yi-Chern Ahmadian, Arash Hosking, Tom Lee, Honglak Wang, Lu Üstün, Ahmet Sherborne, Tom Gallé, Matthias Computation and Language Artificial Intelligence Model merging has shown great promise at combining expert models, but the benefit of merging is unclear when merging "generalist" models trained on many tasks. We explore merging in the context of large (~100B) models, by recycling checkpoints that exhibit tradeoffs among different tasks. Such checkpoints are often created in the process of developing a frontier model, and the suboptimal ones are usually discarded. Given a pool of model checkpoints obtained from different training runs (e.g., different stages, objectives, hyperparameters, and data mixtures), which naturally show tradeoffs across different language capabilities (e.g., instruction following vs. code generation), we investigate whether merging can recycle such suboptimal models into a Pareto-optimal one. Our optimization algorithm tunes the weight of each checkpoint in a linear combination, resulting in such an optimal model that outperforms both individual models and merge-based baselines. Further analysis shows that good merges tend to include almost all checkpoints with non-zero weights, indicating that even seemingly bad initial checkpoints can contribute to good final merges. |
| title | If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.04144 |