MERGE$^3$: Efficient Evolutionary Merging on Consumer-grade GPUs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912367061762048 |
|---|---|
| author | Mencattini, Tommaso Minut, Adrian Robert Crisostomi, Donato Santilli, Andrea Rodolà, Emanuele |
| author_facet | Mencattini, Tommaso Minut, Adrian Robert Crisostomi, Donato Santilli, Andrea Rodolà, Emanuele |
| contents | Evolutionary model merging enables the creation of high-performing multi-task models but remains computationally prohibitive for consumer hardware. We introduce MERGE$^3$, an efficient framework that makes evolutionary merging feasible on a single GPU by reducing fitness computation costs 50$\times$ while preserving performance. MERGE$^3$ achieves this by Extracting a reduced dataset for evaluation, Estimating model abilities using Item Response Theory (IRT), and Evolving optimal merges via IRT-based performance estimators. Our method enables state-of-the-art multilingual and cross-lingual merging, transferring knowledge across languages with significantly lower computational overhead. We provide theoretical guarantees and an open-source library, democratizing high-quality model merging. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_10436 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MERGE$^3$: Efficient Evolutionary Merging on Consumer-grade GPUs Mencattini, Tommaso Minut, Adrian Robert Crisostomi, Donato Santilli, Andrea Rodolà, Emanuele Neural and Evolutionary Computing Artificial Intelligence Machine Learning Evolutionary model merging enables the creation of high-performing multi-task models but remains computationally prohibitive for consumer hardware. We introduce MERGE$^3$, an efficient framework that makes evolutionary merging feasible on a single GPU by reducing fitness computation costs 50$\times$ while preserving performance. MERGE$^3$ achieves this by Extracting a reduced dataset for evaluation, Estimating model abilities using Item Response Theory (IRT), and Evolving optimal merges via IRT-based performance estimators. Our method enables state-of-the-art multilingual and cross-lingual merging, transferring knowledge across languages with significantly lower computational overhead. We provide theoretical guarantees and an open-source library, democratizing high-quality model merging. |
| title | MERGE$^3$: Efficient Evolutionary Merging on Consumer-grade GPUs |
| topic | Neural and Evolutionary Computing Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2502.10436 |