Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2601.04426 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910256085336064 |
|---|---|
| author | Li, Linzhang Dong, Yixin Wang, Guanjie Xu, Ziyi Jiang, Alexander Chen, Tianqi |
| author_facet | Li, Linzhang Dong, Yixin Wang, Guanjie Xu, Ziyi Jiang, Alexander Chen, Tianqi |
| contents | Modern LLM agents increasingly rely on dynamic structured generation, such as tool calling and response protocols. Unlike traditional structured generation with static structures, these workloads vary both across requests and within a request, posing new challenges to existing engines. We present XGrammar-2, a structured generation engine for dynamic agentic workloads. Our design is based on two key ideas: first-class support for tag-triggered structure switching, and fine-grained reuse across requests with different output structures. Concretely, XGrammar-2 introduces TagDispatch for dynamic structural dispatching and Cross-Grammar Cache for substructure-level cache reuse across grammars. It further improves efficiency with an Earley-based adaptive token mask cache, just-in-time compilation, and repetition state compression. Experiments show that XGrammar-2 achieves over 6x faster compilation than prior structured generation engines, and incurs near-zero end-to-end overhead in modern LLM serving systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_04426 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | XGrammar-2: Efficient Dynamic Structured Generation Engine for Agentic LLMs Li, Linzhang Dong, Yixin Wang, Guanjie Xu, Ziyi Jiang, Alexander Chen, Tianqi Artificial Intelligence I.2.11 Modern LLM agents increasingly rely on dynamic structured generation, such as tool calling and response protocols. Unlike traditional structured generation with static structures, these workloads vary both across requests and within a request, posing new challenges to existing engines. We present XGrammar-2, a structured generation engine for dynamic agentic workloads. Our design is based on two key ideas: first-class support for tag-triggered structure switching, and fine-grained reuse across requests with different output structures. Concretely, XGrammar-2 introduces TagDispatch for dynamic structural dispatching and Cross-Grammar Cache for substructure-level cache reuse across grammars. It further improves efficiency with an Earley-based adaptive token mask cache, just-in-time compilation, and repetition state compression. Experiments show that XGrammar-2 achieves over 6x faster compilation than prior structured generation engines, and incurs near-zero end-to-end overhead in modern LLM serving systems. |
| title | XGrammar-2: Efficient Dynamic Structured Generation Engine for Agentic LLMs |
| topic | Artificial Intelligence I.2.11 |
| url | https://arxiv.org/abs/2601.04426 |