AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910672979230720 |
|---|---|
| author | Kim, Dongkyu Kim, Byoungwook Han, Donggeon Eibich, Matouš |
| author_facet | Kim, Dongkyu Kim, Byoungwook Han, Donggeon Eibich, Matouš |
| contents | Using LLMs (Large Language Models) in conjunction with external documents has made RAG (Retrieval-Augmented Generation) an essential technology. Numerous techniques and modules for RAG are being researched, but their performance can vary across different datasets. Finding RAG modules that perform well on specific datasets is challenging. In this paper, we propose the AutoRAG framework, which automatically identifies suitable RAG modules for a given dataset. AutoRAG explores and approximates the optimal combination of RAG modules for the dataset. Additionally, we share the results of optimizing a dataset using AutoRAG. All experimental results and data are publicly available and can be accessed through our GitHub repository https://github.com/Marker-Inc-Korea/AutoRAG_ARAGOG_Paper . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_20878 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline Kim, Dongkyu Kim, Byoungwook Han, Donggeon Eibich, Matouš Computation and Language H.4.0 Using LLMs (Large Language Models) in conjunction with external documents has made RAG (Retrieval-Augmented Generation) an essential technology. Numerous techniques and modules for RAG are being researched, but their performance can vary across different datasets. Finding RAG modules that perform well on specific datasets is challenging. In this paper, we propose the AutoRAG framework, which automatically identifies suitable RAG modules for a given dataset. AutoRAG explores and approximates the optimal combination of RAG modules for the dataset. Additionally, we share the results of optimizing a dataset using AutoRAG. All experimental results and data are publicly available and can be accessed through our GitHub repository https://github.com/Marker-Inc-Korea/AutoRAG_ARAGOG_Paper . |
| title | AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline |
| topic | Computation and Language H.4.0 |
| url | https://arxiv.org/abs/2410.20878 |