ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912921151340544 |
|---|---|
| author | He, Andre Weir, Nathaniel Bostrom, Kaj Nie, Allen Cassel, Darion Bayless, Sam Rangwala, Huzefa |
| author_facet | He, Andre Weir, Nathaniel Bostrom, Kaj Nie, Allen Cassel, Darion Bayless, Sam Rangwala, Huzefa |
| contents | Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising approach for training reasoning language models (RLMs) by leveraging supervision from verifiers. Although verifier implementation is easier than solution annotation for many tasks, existing synthetic data generation methods remain largely solution-centric, while verifier-based methods rely on a few hand-crafted procedural environments. In this work, we scale RLVR by introducing ReSyn, a pipeline that generates diverse reasoning environments equipped with instance generators and verifiers, covering tasks such as constraint satisfaction, algorithmic puzzles, and spatial reasoning. A Qwen2.5-7B-Instruct model trained with RL on ReSyn data achieves consistent gains across reasoning benchmarks and out-of-domain math benchmarks, including a 27\% relative improvement on the challenging BBEH benchmark. Ablations show that verifier-based supervision and increased task diversity both contribute significantly, providing empirical evidence that generating reasoning environments at scale can enhance reasoning abilities in RLMs |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_20117 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Models He, Andre Weir, Nathaniel Bostrom, Kaj Nie, Allen Cassel, Darion Bayless, Sam Rangwala, Huzefa Artificial Intelligence Machine Learning Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising approach for training reasoning language models (RLMs) by leveraging supervision from verifiers. Although verifier implementation is easier than solution annotation for many tasks, existing synthetic data generation methods remain largely solution-centric, while verifier-based methods rely on a few hand-crafted procedural environments. In this work, we scale RLVR by introducing ReSyn, a pipeline that generates diverse reasoning environments equipped with instance generators and verifiers, covering tasks such as constraint satisfaction, algorithmic puzzles, and spatial reasoning. A Qwen2.5-7B-Instruct model trained with RL on ReSyn data achieves consistent gains across reasoning benchmarks and out-of-domain math benchmarks, including a 27\% relative improvement on the challenging BBEH benchmark. Ablations show that verifier-based supervision and increased task diversity both contribute significantly, providing empirical evidence that generating reasoning environments at scale can enhance reasoning abilities in RLMs |
| title | ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Models |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2602.20117 |