QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917370583318528 |
|---|---|
| author | Li, Jiazheng Lin, Hongzhou Lu, Hong Wen, Kaiyue Yang, Zaiwen Gao, Jiaxuan Wu, Yi Zhang, Jingzhao |
| author_facet | Li, Jiazheng Lin, Hongzhou Lu, Hong Wen, Kaiyue Yang, Zaiwen Gao, Jiaxuan Wu, Yi Zhang, Jingzhao |
| contents | Reinforcement learning (RL) has emerged as a central paradigm for training large language models (LLMs) in reasoning tasks. Yet recent studies question RL's ability to incentivize reasoning capacity beyond the base model. This raises a key challenge: how can RL be adapted to solve harder reasoning problems more effectively? To address this challenge, we propose a simple yet effective strategy via Question Augmentation: introduce partial solutions during training to reduce problem difficulty and provide more informative learning signals. Our method, QuestA, when applied during RL training on math reasoning tasks, not only improves pass@1 but also pass@k-particularly on problems where standard RL struggles to make progress. This enables continual improvement over strong open-source models such as DeepScaleR and OpenMath Nemotron, further enhancing their reasoning capabilities. We achieve new state-of-the-art results on math benchmarks using 1.5B-parameter models: 72.50% (+10.73%) on AIME24, 62.29% (+12.79%) on AIME25, and 41.67% (+10.11%) on HMMT25. Code, data and model are available at https://github.com/foreverlasting1202/QuestA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_13266 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation Li, Jiazheng Lin, Hongzhou Lu, Hong Wen, Kaiyue Yang, Zaiwen Gao, Jiaxuan Wu, Yi Zhang, Jingzhao Computation and Language Artificial Intelligence Reinforcement learning (RL) has emerged as a central paradigm for training large language models (LLMs) in reasoning tasks. Yet recent studies question RL's ability to incentivize reasoning capacity beyond the base model. This raises a key challenge: how can RL be adapted to solve harder reasoning problems more effectively? To address this challenge, we propose a simple yet effective strategy via Question Augmentation: introduce partial solutions during training to reduce problem difficulty and provide more informative learning signals. Our method, QuestA, when applied during RL training on math reasoning tasks, not only improves pass@1 but also pass@k-particularly on problems where standard RL struggles to make progress. This enables continual improvement over strong open-source models such as DeepScaleR and OpenMath Nemotron, further enhancing their reasoning capabilities. We achieve new state-of-the-art results on math benchmarks using 1.5B-parameter models: 72.50% (+10.73%) on AIME24, 62.29% (+12.79%) on AIME25, and 41.67% (+10.11%) on HMMT25. Code, data and model are available at https://github.com/foreverlasting1202/QuestA. |
| title | QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2507.13266 |