Text-to-SQL Domain Adaptation via Human-LLM Collaborative Data Annotation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915617108393984 |
|---|---|
| author | Tian, Yuan Lee, Daniel Wu, Fei Mai, Tung Qian, Kun Sahai, Siddhartha Zhang, Tianyi Li, Yunyao |
| author_facet | Tian, Yuan Lee, Daniel Wu, Fei Mai, Tung Qian, Kun Sahai, Siddhartha Zhang, Tianyi Li, Yunyao |
| contents | Text-to-SQL models, which parse natural language (NL) questions to executable SQL queries, are increasingly adopted in real-world applications. However, deploying such models in the real world often requires adapting them to the highly specialized database schemas used in specific applications. We find that existing text-to-SQL models experience significant performance drops when applied to new schemas, primarily due to the lack of domain-specific data for fine-tuning. This data scarcity also limits the ability to effectively evaluate model performance in new domains. Continuously obtaining high-quality text-to-SQL data for evolving schemas is prohibitively expensive in real-world scenarios. To bridge this gap, we propose SQLsynth, a human-in-the-loop text-to-SQL data annotation system. SQLsynth streamlines the creation of high-quality text-to-SQL datasets through human-LLM collaboration in a structured workflow. A within-subjects user study comparing SQLsynth with manual annotation and ChatGPT shows that SQLsynth significantly accelerates text-to-SQL data annotation, reduces cognitive load, and produces datasets that are more accurate, natural, and diverse. Our code is available at https://github.com/magic-YuanTian/SQLsynth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15980 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Text-to-SQL Domain Adaptation via Human-LLM Collaborative Data Annotation Tian, Yuan Lee, Daniel Wu, Fei Mai, Tung Qian, Kun Sahai, Siddhartha Zhang, Tianyi Li, Yunyao Human-Computer Interaction Artificial Intelligence Databases Text-to-SQL models, which parse natural language (NL) questions to executable SQL queries, are increasingly adopted in real-world applications. However, deploying such models in the real world often requires adapting them to the highly specialized database schemas used in specific applications. We find that existing text-to-SQL models experience significant performance drops when applied to new schemas, primarily due to the lack of domain-specific data for fine-tuning. This data scarcity also limits the ability to effectively evaluate model performance in new domains. Continuously obtaining high-quality text-to-SQL data for evolving schemas is prohibitively expensive in real-world scenarios. To bridge this gap, we propose SQLsynth, a human-in-the-loop text-to-SQL data annotation system. SQLsynth streamlines the creation of high-quality text-to-SQL datasets through human-LLM collaboration in a structured workflow. A within-subjects user study comparing SQLsynth with manual annotation and ChatGPT shows that SQLsynth significantly accelerates text-to-SQL data annotation, reduces cognitive load, and produces datasets that are more accurate, natural, and diverse. Our code is available at https://github.com/magic-YuanTian/SQLsynth. |
| title | Text-to-SQL Domain Adaptation via Human-LLM Collaborative Data Annotation |
| topic | Human-Computer Interaction Artificial Intelligence Databases |
| url | https://arxiv.org/abs/2502.15980 |