When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challenges
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910861757513728 |
|---|---|
| author | Wang, Chao Zhao, Jiaxuan Jiao, Licheng Li, Lingling Liu, Fang Yang, Shuyuan |
| author_facet | Wang, Chao Zhao, Jiaxuan Jiao, Licheng Li, Lingling Liu, Fang Yang, Shuyuan |
| contents | Pre-trained large language models (LLMs) exhibit powerful capabilities for generating natural text. Evolutionary algorithms (EAs) can discover diverse solutions to complex real-world problems. Motivated by the common collective and directionality of text generation and evolution, this paper first illustrates the conceptual parallels between LLMs and EAs at a micro level, which includes multiple one-to-one key characteristics: token representation and individual representation, position encoding and fitness shaping, position embedding and selection, Transformers block and reproduction, and model training and parameter adaptation. These parallels highlight potential opportunities for technical advancements in both LLMs and EAs. Subsequently, we analyze existing interdisciplinary research from a macro perspective to uncover critical challenges, with a particular focus on evolutionary fine-tuning and LLM-enhanced EAs. These analyses not only provide insights into the evolutionary mechanisms behind LLMs but also offer potential directions for enhancing the capabilities of artificial agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_10510 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challenges Wang, Chao Zhao, Jiaxuan Jiao, Licheng Li, Lingling Liu, Fang Yang, Shuyuan Neural and Evolutionary Computing Artificial Intelligence Computation and Language Machine Learning Pre-trained large language models (LLMs) exhibit powerful capabilities for generating natural text. Evolutionary algorithms (EAs) can discover diverse solutions to complex real-world problems. Motivated by the common collective and directionality of text generation and evolution, this paper first illustrates the conceptual parallels between LLMs and EAs at a micro level, which includes multiple one-to-one key characteristics: token representation and individual representation, position encoding and fitness shaping, position embedding and selection, Transformers block and reproduction, and model training and parameter adaptation. These parallels highlight potential opportunities for technical advancements in both LLMs and EAs. Subsequently, we analyze existing interdisciplinary research from a macro perspective to uncover critical challenges, with a particular focus on evolutionary fine-tuning and LLM-enhanced EAs. These analyses not only provide insights into the evolutionary mechanisms behind LLMs but also offer potential directions for enhancing the capabilities of artificial agents. |
| title | When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challenges |
| topic | Neural and Evolutionary Computing Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2401.10510 |