Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.27532 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912679859322880 |
|---|---|
| author | Srikanth, Neha Bursztyn, Victor Mathur, Puneet Nenkova, Ani |
| author_facet | Srikanth, Neha Bursztyn, Victor Mathur, Puneet Nenkova, Ani |
| contents | We introduce SQLSpace, a human-interpretable, generalizable, compact representation for text-to-SQL examples derived with minimal human intervention. We demonstrate the utility of these representations in evaluation with three use cases: (i) closely comparing and contrasting the composition of popular text-to-SQL benchmarks to identify unique dimensions of examples they evaluate, (ii) understanding model performance at a granular level beyond overall accuracy scores, and (iii) improving model performance through targeted query rewriting based on learned correctness estimation. We show that SQLSpace enables analysis that would be difficult with raw examples alone: it reveals compositional differences between benchmarks, exposes performance patterns obscured by accuracy alone, and supports modeling of query success. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_27532 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps Srikanth, Neha Bursztyn, Victor Mathur, Puneet Nenkova, Ani Computation and Language We introduce SQLSpace, a human-interpretable, generalizable, compact representation for text-to-SQL examples derived with minimal human intervention. We demonstrate the utility of these representations in evaluation with three use cases: (i) closely comparing and contrasting the composition of popular text-to-SQL benchmarks to identify unique dimensions of examples they evaluate, (ii) understanding model performance at a granular level beyond overall accuracy scores, and (iii) improving model performance through targeted query rewriting based on learned correctness estimation. We show that SQLSpace enables analysis that would be difficult with raw examples alone: it reveals compositional differences between benchmarks, exposes performance patterns obscured by accuracy alone, and supports modeling of query success. |
| title | SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.27532 |