ConstrainedSQL: Training LLMs for Text2SQL via Constrained Reinforcement Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915613261168640 |
|---|---|
| author | Chen, Weiqin Pham, Nhan Huu Glass, Michael Robert Vu, Long Hai Rossiello, Gaetano Subramanian, Dharmashankar Paternain, Santiago |
| author_facet | Chen, Weiqin Pham, Nhan Huu Glass, Michael Robert Vu, Long Hai Rossiello, Gaetano Subramanian, Dharmashankar Paternain, Santiago |
| contents | Reinforcement learning (RL) has demonstrated significant promise in enhancing the reasoning capabilities of Text2SQL LLMs, especially with advanced algorithms such as GRPO and DAPO. However, the performance of these methods is highly sensitive to the design of reward functions. Inappropriate rewards can lead to reward hacking, where models exploit loopholes in the reward structure to achieve high scores without genuinely solving the task. This work considers a constrained RL framework for Text2SQL that incorporates natural and interpretable reward and constraint signals, while dynamically balancing trade-offs among them during the training. We establish the theoretical guarantees of our constrained RL framework and our numerical experiments on the well-known Text2SQL datasets substantiate the improvement of our approach over the state-of-the-art RL-trained LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_09693 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ConstrainedSQL: Training LLMs for Text2SQL via Constrained Reinforcement Learning Chen, Weiqin Pham, Nhan Huu Glass, Michael Robert Vu, Long Hai Rossiello, Gaetano Subramanian, Dharmashankar Paternain, Santiago Machine Learning Reinforcement learning (RL) has demonstrated significant promise in enhancing the reasoning capabilities of Text2SQL LLMs, especially with advanced algorithms such as GRPO and DAPO. However, the performance of these methods is highly sensitive to the design of reward functions. Inappropriate rewards can lead to reward hacking, where models exploit loopholes in the reward structure to achieve high scores without genuinely solving the task. This work considers a constrained RL framework for Text2SQL that incorporates natural and interpretable reward and constraint signals, while dynamically balancing trade-offs among them during the training. We establish the theoretical guarantees of our constrained RL framework and our numerical experiments on the well-known Text2SQL datasets substantiate the improvement of our approach over the state-of-the-art RL-trained LLMs. |
| title | ConstrainedSQL: Training LLMs for Text2SQL via Constrained Reinforcement Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2511.09693 |