Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908495846047744 |
|---|---|
| author | Lupidi, Alisia Gemmell, Carlos Cancedda, Nicola Dwivedi-Yu, Jane Weston, Jason Foerster, Jakob Raileanu, Roberta Lomeli, Maria |
| author_facet | Lupidi, Alisia Gemmell, Carlos Cancedda, Nicola Dwivedi-Yu, Jane Weston, Jason Foerster, Jakob Raileanu, Roberta Lomeli, Maria |
| contents | Synthetic data generation has recently emerged as a promising approach for enhancing the capabilities of large language models (LLMs) without the need for expensive human annotations. However, existing methods often generate data that can be low quality or contrived. In this paper, we introduce Source2Synth, a scalable approach for synthetic data generation and curation that is grounded in real-world data sources. Source2Synth takes as input a custom data source and produces synthetic data examples with intermediate reasoning steps. Our method improves the dataset quality by discarding low-quality generations based on their answerability. We demonstrate the generality of this approach by applying it to two tasks that leverage two different types of data: multi-hop question answering (MHQA), where we test complex reasoning abilities leveraging documents, and tabular question answering (TQA), where we test tool usage leveraging tables. Our method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on HotpotQA compared to the fine-tuned baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_08239 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources Lupidi, Alisia Gemmell, Carlos Cancedda, Nicola Dwivedi-Yu, Jane Weston, Jason Foerster, Jakob Raileanu, Roberta Lomeli, Maria Computation and Language Artificial Intelligence Synthetic data generation has recently emerged as a promising approach for enhancing the capabilities of large language models (LLMs) without the need for expensive human annotations. However, existing methods often generate data that can be low quality or contrived. In this paper, we introduce Source2Synth, a scalable approach for synthetic data generation and curation that is grounded in real-world data sources. Source2Synth takes as input a custom data source and produces synthetic data examples with intermediate reasoning steps. Our method improves the dataset quality by discarding low-quality generations based on their answerability. We demonstrate the generality of this approach by applying it to two tasks that leverage two different types of data: multi-hop question answering (MHQA), where we test complex reasoning abilities leveraging documents, and tabular question answering (TQA), where we test tool usage leveraging tables. Our method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on HotpotQA compared to the fine-tuned baselines. |
| title | Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.08239 |