Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.20878 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910672979230720 |
|---|---|
| author | Kim, Dongkyu Kim, Byoungwook Han, Donggeon Eibich, Matouš |
| author_facet | Kim, Dongkyu Kim, Byoungwook Han, Donggeon Eibich, Matouš |
| contents | Using LLMs (Large Language Models) in conjunction with external documents has made RAG (Retrieval-Augmented Generation) an essential technology. Numerous techniques and modules for RAG are being researched, but their performance can vary across different datasets. Finding RAG modules that perform well on specific datasets is challenging. In this paper, we propose the AutoRAG framework, which automatically identifies suitable RAG modules for a given dataset. AutoRAG explores and approximates the optimal combination of RAG modules for the dataset. Additionally, we share the results of optimizing a dataset using AutoRAG. All experimental results and data are publicly available and can be accessed through our GitHub repository https://github.com/Marker-Inc-Korea/AutoRAG_ARAGOG_Paper . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_20878 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline Kim, Dongkyu Kim, Byoungwook Han, Donggeon Eibich, Matouš Computation and Language H.4.0 Using LLMs (Large Language Models) in conjunction with external documents has made RAG (Retrieval-Augmented Generation) an essential technology. Numerous techniques and modules for RAG are being researched, but their performance can vary across different datasets. Finding RAG modules that perform well on specific datasets is challenging. In this paper, we propose the AutoRAG framework, which automatically identifies suitable RAG modules for a given dataset. AutoRAG explores and approximates the optimal combination of RAG modules for the dataset. Additionally, we share the results of optimizing a dataset using AutoRAG. All experimental results and data are publicly available and can be accessed through our GitHub repository https://github.com/Marker-Inc-Korea/AutoRAG_ARAGOG_Paper . |
| title | AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline |
| topic | Computation and Language H.4.0 |
| url | https://arxiv.org/abs/2410.20878 |