EvoMerge: Neuroevolution for Large Language Models
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911768748490752 |
|---|---|
| author | Jiang, Yushu |
| author_facet | Jiang, Yushu |
| contents | Extensive fine-tuning on Large Language Models does not always yield better results. Oftentimes, models tend to get better at imitating one form of data without gaining greater reasoning ability and may even end up losing some intelligence. Here I introduce EvoMerge, a systematic approach to large language model training and merging. Leveraging model merging for weight crossover and fine-tuning for weight mutation, EvoMerge establishes an evolutionary process aimed at pushing models beyond the limits of conventional fine-tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00070 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EvoMerge: Neuroevolution for Large Language Models Jiang, Yushu Neural and Evolutionary Computing Artificial Intelligence Computation and Language Machine Learning Extensive fine-tuning on Large Language Models does not always yield better results. Oftentimes, models tend to get better at imitating one form of data without gaining greater reasoning ability and may even end up losing some intelligence. Here I introduce EvoMerge, a systematic approach to large language model training and merging. Leveraging model merging for weight crossover and fine-tuning for weight mutation, EvoMerge establishes an evolutionary process aimed at pushing models beyond the limits of conventional fine-tuning. |
| title | EvoMerge: Neuroevolution for Large Language Models |
| topic | Neural and Evolutionary Computing Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2402.00070 |