Synthesizing Text-to-SQL Data from Weak and Strong LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916348496445440 |
|---|---|
| author | Yang, Jiaxi Hui, Binyuan Yang, Min Yang, Jian Lin, Junyang Zhou, Chang |
| author_facet | Yang, Jiaxi Hui, Binyuan Yang, Min Yang, Jian Lin, Junyang Zhou, Chang |
| contents | The capability gap between open-source and closed-source large language models (LLMs) remains a challenge in text-to-SQL tasks. In this paper, we introduce a synthetic data approach that combines data produced by larger, more powerful models (strong models) with error information data generated by smaller, not well-aligned models (weak models). The method not only enhances the domain generalization of text-to-SQL models but also explores the potential of error data supervision through preference learning. Furthermore, we employ the synthetic data approach for instruction tuning on open-source LLMs, resulting SENSE, a specialized text-to-SQL model. The effectiveness of SENSE is demonstrated through state-of-the-art results on the SPIDER and BIRD benchmarks, bridging the performance gap between open-source models and methods prompted by closed-source models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_03256 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Synthesizing Text-to-SQL Data from Weak and Strong LLMs Yang, Jiaxi Hui, Binyuan Yang, Min Yang, Jian Lin, Junyang Zhou, Chang Computation and Language The capability gap between open-source and closed-source large language models (LLMs) remains a challenge in text-to-SQL tasks. In this paper, we introduce a synthetic data approach that combines data produced by larger, more powerful models (strong models) with error information data generated by smaller, not well-aligned models (weak models). The method not only enhances the domain generalization of text-to-SQL models but also explores the potential of error data supervision through preference learning. Furthermore, we employ the synthetic data approach for instruction tuning on open-source LLMs, resulting SENSE, a specialized text-to-SQL model. The effectiveness of SENSE is demonstrated through state-of-the-art results on the SPIDER and BIRD benchmarks, bridging the performance gap between open-source models and methods prompted by closed-source models. |
| title | Synthesizing Text-to-SQL Data from Weak and Strong LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.03256 |