Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917870884093952 |
|---|---|
| author | Liu, Geling Tan, Yunzhi Zhong, Ruichao Xie, Yuanzhen Zhao, Lingchen Wang, Qian Hu, Bo Li, Zang |
| author_facet | Liu, Geling Tan, Yunzhi Zhong, Ruichao Xie, Yuanzhen Zhao, Lingchen Wang, Qian Hu, Bo Li, Zang |
| contents | Recently, large language models (LLMs) have significantly improved the performance of text-to-SQL systems. Nevertheless, many state-of-the-art (SOTA) approaches have overlooked the critical aspect of system robustness. Our experiments reveal that while LLM-driven methods excel on standard datasets, their accuracy is notably compromised when faced with adversarial perturbations. To address this challenge, we propose a robust text-to-SQL solution, called Solid-SQL, designed to integrate with various LLMs. We focus on the pre-processing stage, training a robust schema-linking model enhanced by LLM-based data augmentation. Additionally, we design a two-round, structural similarity-based example retrieval strategy for in-context learning. Our method achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% on the general Spider and Bird benchmarks, respectively. Furthermore, experimental results show that Solid-SQL delivers an average improvement of 11.6% compared to baselines on the perturbed Spider-Syn, Spider-Realistic, and Dr. Spider benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_12522 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL Liu, Geling Tan, Yunzhi Zhong, Ruichao Xie, Yuanzhen Zhao, Lingchen Wang, Qian Hu, Bo Li, Zang Computation and Language Artificial Intelligence Recently, large language models (LLMs) have significantly improved the performance of text-to-SQL systems. Nevertheless, many state-of-the-art (SOTA) approaches have overlooked the critical aspect of system robustness. Our experiments reveal that while LLM-driven methods excel on standard datasets, their accuracy is notably compromised when faced with adversarial perturbations. To address this challenge, we propose a robust text-to-SQL solution, called Solid-SQL, designed to integrate with various LLMs. We focus on the pre-processing stage, training a robust schema-linking model enhanced by LLM-based data augmentation. Additionally, we design a two-round, structural similarity-based example retrieval strategy for in-context learning. Our method achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% on the general Spider and Bird benchmarks, respectively. Furthermore, experimental results show that Solid-SQL delivers an average improvement of 11.6% compared to baselines on the perturbed Spider-Syn, Spider-Realistic, and Dr. Spider benchmarks. |
| title | Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.12522 |