Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.11157 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929348267737088 |
|---|---|
| author | Ostapenko, Oleksiy Su, Zhan Ponti, Edoardo Maria Charlin, Laurent Roux, Nicolas Le Pereira, Matheus Caccia, Lucas Sordoni, Alessandro |
| author_facet | Ostapenko, Oleksiy Su, Zhan Ponti, Edoardo Maria Charlin, Laurent Roux, Nicolas Le Pereira, Matheus Caccia, Lucas Sordoni, Alessandro |
| contents | The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trained adapters to improve performance for new tasks. We study how to best build a library of adapters given multi-task data and devise techniques for both zero-shot and supervised task generalization through routing in such library. We benchmark existing approaches to build this library and introduce model-based clustering, MBC, a method that groups tasks based on the similarity of their adapter parameters, indirectly optimizing for transfer across the multi-task dataset. To re-use the library, we present a novel zero-shot routing mechanism, Arrow, which enables dynamic selection of the most relevant adapters for new inputs without the need for retraining. We experiment with several LLMs, such as Phi-2 and Mistral, on a wide array of held-out tasks, verifying that MBC-based adapters and Arrow routing lead to superior generalization to new tasks. We make steps towards creating modular, adaptable LLMs that can match or outperform traditional joint training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11157 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Modular LLMs by Building and Reusing a Library of LoRAs Ostapenko, Oleksiy Su, Zhan Ponti, Edoardo Maria Charlin, Laurent Roux, Nicolas Le Pereira, Matheus Caccia, Lucas Sordoni, Alessandro Machine Learning Computation and Language The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trained adapters to improve performance for new tasks. We study how to best build a library of adapters given multi-task data and devise techniques for both zero-shot and supervised task generalization through routing in such library. We benchmark existing approaches to build this library and introduce model-based clustering, MBC, a method that groups tasks based on the similarity of their adapter parameters, indirectly optimizing for transfer across the multi-task dataset. To re-use the library, we present a novel zero-shot routing mechanism, Arrow, which enables dynamic selection of the most relevant adapters for new inputs without the need for retraining. We experiment with several LLMs, such as Phi-2 and Mistral, on a wide array of held-out tasks, verifying that MBC-based adapters and Arrow routing lead to superior generalization to new tasks. We make steps towards creating modular, adaptable LLMs that can match or outperform traditional joint training. |
| title | Towards Modular LLMs by Building and Reusing a Library of LoRAs |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2405.11157 |