Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916433473044480 |
|---|---|
| author | Gauthier-Caron, Thomas Siriwardhana, Shamane Stein, Elliot Ehghaghi, Malikeh Goddard, Charles McQuade, Mark Solawetz, Jacob Labonne, Maxime |
| author_facet | Gauthier-Caron, Thomas Siriwardhana, Shamane Stein, Elliot Ehghaghi, Malikeh Goddard, Charles McQuade, Mark Solawetz, Jacob Labonne, Maxime |
| contents | By merging models, AI systems can combine the distinct strengths of separate language models, achieving a balance between multiple capabilities without requiring substantial retraining. However, the integration process can be intricate due to differences in training methods and fine-tuning, typically necessitating specialized knowledge and repeated refinement. This paper explores model merging techniques across a spectrum of complexity, examining where automated methods like evolutionary strategies stand compared to hyperparameter-driven approaches such as DARE, TIES-Merging and simpler methods like Model Soups. In addition, we introduce Differentiable Adaptive Merging (DAM), an efficient, adaptive merging approach as an alternative to evolutionary merging that optimizes model integration through scaling coefficients, minimizing computational demands. Our findings reveal that even simple averaging methods, like Model Soups, perform competitively when model similarity is high, underscoring each technique's unique strengths and limitations. We open-sourced DAM, including the implementation code and experiment pipeline, on GitHub: https://github.com/arcee-ai/DAM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08371 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation Gauthier-Caron, Thomas Siriwardhana, Shamane Stein, Elliot Ehghaghi, Malikeh Goddard, Charles McQuade, Mark Solawetz, Jacob Labonne, Maxime Computation and Language Artificial Intelligence Machine Learning By merging models, AI systems can combine the distinct strengths of separate language models, achieving a balance between multiple capabilities without requiring substantial retraining. However, the integration process can be intricate due to differences in training methods and fine-tuning, typically necessitating specialized knowledge and repeated refinement. This paper explores model merging techniques across a spectrum of complexity, examining where automated methods like evolutionary strategies stand compared to hyperparameter-driven approaches such as DARE, TIES-Merging and simpler methods like Model Soups. In addition, we introduce Differentiable Adaptive Merging (DAM), an efficient, adaptive merging approach as an alternative to evolutionary merging that optimizes model integration through scaling coefficients, minimizing computational demands. Our findings reveal that even simple averaging methods, like Model Soups, perform competitively when model similarity is high, underscoring each technique's unique strengths and limitations. We open-sourced DAM, including the implementation code and experiment pipeline, on GitHub: https://github.com/arcee-ai/DAM. |
| title | Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2410.08371 |