Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.13130 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909039629172736 |
|---|---|
| author | Li, Junjie Wang, Ziao Ma, NingXuan Ma, Jianghong Zhang, Xiaofeng |
| author_facet | Li, Junjie Wang, Ziao Ma, NingXuan Ma, Jianghong Zhang, Xiaofeng |
| contents | Existing reasoning data curation pipelines score whole samples, treating every intermediate step as equally valuable. In reality, steps within a trace contribute very unevenly, and selecting reasoning data well requires assessing them individually. We present GRACE, a gradient-aligned curation method that views each reasoning trace as a sequence of optimization events and scores every step by two complementary signals: its alignment with the answer-oriented gradient direction, and its consistency with the preceding reasoning trajectory. Step-level scores are aggregated into a sample-level value for subset selection, using only the model's internal optimization signals and no external reward models or step annotations. To make this scalable, GRACE introduces a representation-level gradient proxy that estimates step-level alignment from token-level upstream signals in a single forward pass. Post-training Qwen3-VL-2B-Instruct on MMathCoT-1M, GRACE reaches 108.8% of the full-data performance with 20% of the data and retains 100.2% with only 5%, with subsets that transfer effectively across model backbones. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_13130 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training Li, Junjie Wang, Ziao Ma, NingXuan Ma, Jianghong Zhang, Xiaofeng Artificial Intelligence Existing reasoning data curation pipelines score whole samples, treating every intermediate step as equally valuable. In reality, steps within a trace contribute very unevenly, and selecting reasoning data well requires assessing them individually. We present GRACE, a gradient-aligned curation method that views each reasoning trace as a sequence of optimization events and scores every step by two complementary signals: its alignment with the answer-oriented gradient direction, and its consistency with the preceding reasoning trajectory. Step-level scores are aggregated into a sample-level value for subset selection, using only the model's internal optimization signals and no external reward models or step annotations. To make this scalable, GRACE introduces a representation-level gradient proxy that estimates step-level alignment from token-level upstream signals in a single forward pass. Post-training Qwen3-VL-2B-Instruct on MMathCoT-1M, GRACE reaches 108.8% of the full-data performance with 20% of the data and retains 100.2% with only 5%, with subsets that transfer effectively across model backbones. |
| title | GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2605.13130 |