Continual Learning of Domain Knowledge from Human Feedback in Text-to-SQL
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909930256072704 |
|---|---|
| author | Cook, Thomas Patel, Kelly Vellaichamy, Sivapriya Sehwag, Udari Madhushani Rahimi, Saba Zeng, Zhen Ganesh, Sumitra |
| author_facet | Cook, Thomas Patel, Kelly Vellaichamy, Sivapriya Sehwag, Udari Madhushani Rahimi, Saba Zeng, Zhen Ganesh, Sumitra |
| contents | Large Language Models (LLMs) can generate SQL queries from natural language questions but struggle with database-specific schemas and tacit domain knowledge. We introduce a framework for continual learning from human feedback in text-to-SQL, where a learning agent receives natural language feedback to refine queries and distills the revealed knowledge for reuse on future tasks. This distilled knowledge is stored in a structured memory, enabling the agent to improve execution accuracy over time. We design and evaluate multiple variations of a learning agent architecture that vary in how they capture and retrieve past experiences. Experiments on the BIRD benchmark Dev set show that memory-augmented agents, particularly the Procedural Agent, achieve significant accuracy gains and error reduction by leveraging human-in-the-loop feedback. Our results highlight the importance of transforming tacit human expertise into reusable knowledge, paving the way for more adaptive, domain-aware text-to-SQL systems that continually learn from a human-in-the-loop. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_10674 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Continual Learning of Domain Knowledge from Human Feedback in Text-to-SQL Cook, Thomas Patel, Kelly Vellaichamy, Sivapriya Sehwag, Udari Madhushani Rahimi, Saba Zeng, Zhen Ganesh, Sumitra Computation and Language Artificial Intelligence Databases 68T05 Large Language Models (LLMs) can generate SQL queries from natural language questions but struggle with database-specific schemas and tacit domain knowledge. We introduce a framework for continual learning from human feedback in text-to-SQL, where a learning agent receives natural language feedback to refine queries and distills the revealed knowledge for reuse on future tasks. This distilled knowledge is stored in a structured memory, enabling the agent to improve execution accuracy over time. We design and evaluate multiple variations of a learning agent architecture that vary in how they capture and retrieve past experiences. Experiments on the BIRD benchmark Dev set show that memory-augmented agents, particularly the Procedural Agent, achieve significant accuracy gains and error reduction by leveraging human-in-the-loop feedback. Our results highlight the importance of transforming tacit human expertise into reusable knowledge, paving the way for more adaptive, domain-aware text-to-SQL systems that continually learn from a human-in-the-loop. |
| title | Continual Learning of Domain Knowledge from Human Feedback in Text-to-SQL |
| topic | Computation and Language Artificial Intelligence Databases 68T05 |
| url | https://arxiv.org/abs/2511.10674 |