EvoGM: Learning to Merge LLMs via Evolutionary Generative Optimization
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913169662803968 |
|---|---|
| author | Jiang, Tao Yu, Xinmeng Yi, Chenhao Wu, Yiling Li, Yan Cheng, Ran Jiang, Dongmei Zhang, Jianguo |
| author_facet | Jiang, Tao Yu, Xinmeng Yi, Chenhao Wu, Yiling Li, Yan Cheng, Ran Jiang, Dongmei Zhang, Jianguo |
| contents | Evolutionary model merging provides a powerful framework for the automated, training-free composition of LLMs through parameter-space search. However, existing methods predominantly rely on stochastic, hand-crafted operators that overlook the underlying performance landscape of the coefficient space. We propose Evolutionary Generative Merging (EvoGM), a framework that transcends manual heuristics by employing learnable generative modeling to optimize merging coefficients. Specifically, EvoGM features a dual-generator architecture with cycle-consistent learning to adaptively sample and refine promising merging candidates. By constructing winner-loser pairs from historical search trajectories, our framework effectively captures high-performance parameter distributions and maximizes data efficiency. This generative process is seamlessly integrated into a multi-round evolutionary pipeline, where elite merged models iteratively serve as new expert foundations. Extensive experiments across diverse benchmarks demonstrate that EvoGM significantly outperforms state-of-the-art baselines, exhibiting robust performance on both seen and unseen tasks. Code and data are available at https://github.com/JiangTao97/evogm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_29295 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | EvoGM: Learning to Merge LLMs via Evolutionary Generative Optimization Jiang, Tao Yu, Xinmeng Yi, Chenhao Wu, Yiling Li, Yan Cheng, Ran Jiang, Dongmei Zhang, Jianguo Neural and Evolutionary Computing Evolutionary model merging provides a powerful framework for the automated, training-free composition of LLMs through parameter-space search. However, existing methods predominantly rely on stochastic, hand-crafted operators that overlook the underlying performance landscape of the coefficient space. We propose Evolutionary Generative Merging (EvoGM), a framework that transcends manual heuristics by employing learnable generative modeling to optimize merging coefficients. Specifically, EvoGM features a dual-generator architecture with cycle-consistent learning to adaptively sample and refine promising merging candidates. By constructing winner-loser pairs from historical search trajectories, our framework effectively captures high-performance parameter distributions and maximizes data efficiency. This generative process is seamlessly integrated into a multi-round evolutionary pipeline, where elite merged models iteratively serve as new expert foundations. Extensive experiments across diverse benchmarks demonstrate that EvoGM significantly outperforms state-of-the-art baselines, exhibiting robust performance on both seen and unseen tasks. Code and data are available at https://github.com/JiangTao97/evogm. |
| title | EvoGM: Learning to Merge LLMs via Evolutionary Generative Optimization |
| topic | Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2605.29295 |