Learning to Generate Structured Output with Schema Reinforcement Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909526760882176 |
|---|---|
| author | Lu, Yaxi Li, Haolun Cong, Xin Zhang, Zhong Wu, Yesai Lin, Yankai Liu, Zhiyuan Liu, Fangming Sun, Maosong |
| author_facet | Lu, Yaxi Li, Haolun Cong, Xin Zhang, Zhong Wu, Yesai Lin, Yankai Liu, Zhiyuan Liu, Fangming Sun, Maosong |
| contents | This study investigates the structured generation capabilities of large language models (LLMs), focusing on producing valid JSON outputs against a given schema. Despite the widespread use of JSON in integrating language models with programs, there is a lack of comprehensive analysis and benchmarking of these capabilities. We explore various aspects of JSON generation, such as structure understanding, escaping, and natural language description, to determine how to assess and enable LLMs to generate valid responses. Building upon this, we propose SchemaBench features around 40K different JSON schemas to obtain and assess models' abilities in generating valid JSON. We find that the latest LLMs are still struggling to generate a valid JSON string. Moreover, we demonstrate that incorporating reinforcement learning with a Fine-grained Schema Validator can further enhance models' understanding of JSON schema, leading to improved performance. Our models demonstrate significant improvement in both generating JSON outputs and downstream tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18878 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning to Generate Structured Output with Schema Reinforcement Learning Lu, Yaxi Li, Haolun Cong, Xin Zhang, Zhong Wu, Yesai Lin, Yankai Liu, Zhiyuan Liu, Fangming Sun, Maosong Computation and Language I.2.7 This study investigates the structured generation capabilities of large language models (LLMs), focusing on producing valid JSON outputs against a given schema. Despite the widespread use of JSON in integrating language models with programs, there is a lack of comprehensive analysis and benchmarking of these capabilities. We explore various aspects of JSON generation, such as structure understanding, escaping, and natural language description, to determine how to assess and enable LLMs to generate valid responses. Building upon this, we propose SchemaBench features around 40K different JSON schemas to obtain and assess models' abilities in generating valid JSON. We find that the latest LLMs are still struggling to generate a valid JSON string. Moreover, we demonstrate that incorporating reinforcement learning with a Fine-grained Schema Validator can further enhance models' understanding of JSON schema, leading to improved performance. Our models demonstrate significant improvement in both generating JSON outputs and downstream tasks. |
| title | Learning to Generate Structured Output with Schema Reinforcement Learning |
| topic | Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2502.18878 |