Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917210130219008 |
|---|---|
| author | Liu, Runxuan Ou, Xianhao Ma, Xinyan Wang, Jiyuan Liang, Jiafeng Li, Jiaqi He, Tao Chu, Zheng Mu, Rongchuan Wang, Zekun Wang, Baoxin Wu, Dayong Liu, Ming Wang, Shijin Hu, Guoping Qin, Bing |
| author_facet | Liu, Runxuan Ou, Xianhao Ma, Xinyan Wang, Jiyuan Liang, Jiafeng Li, Jiaqi He, Tao Chu, Zheng Mu, Rongchuan Wang, Zekun Wang, Baoxin Wu, Dayong Liu, Ming Wang, Shijin Hu, Guoping Qin, Bing |
| contents | Long Chain-of-Thought (LCoT), achieved by Reinforcement Learning with Verifiable Rewards (RLVR), has proven effective in enhancing the reasoning capabilities of Large Language Models (LLMs). However, reasoning in current LLMs is primarily generated as plain text, where performing semantic evaluation on such unstructured data creates a computational bottleneck during training. Despite RLVR-based optimization, existing methods still suffer from coarse-grained supervision, reward hacking, high training costs, and poor generalization. To address these issues, we propose the Graph Reasoning Paradigm (GRP), which realizes structured and symbolic reasoning, implemented via graph-structured representations with step-level cognitive labels. Building upon GRP, we further design Process-Aware Stratified Clipping Group Relative Policy Optimization (PASC-GRPO), which leverages structured evaluation to replace semantic evaluation, achieves process-aware verification through graph-structured outcome rewards, and mitigates reward hacking via stratified clipping advantage estimation. Experiments demonstrate significant improvements across mathematical reasoning and code generation tasks. Data, models, and code will be released later. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_12995 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models Liu, Runxuan Ou, Xianhao Ma, Xinyan Wang, Jiyuan Liang, Jiafeng Li, Jiaqi He, Tao Chu, Zheng Mu, Rongchuan Wang, Zekun Wang, Baoxin Wu, Dayong Liu, Ming Wang, Shijin Hu, Guoping Qin, Bing Computation and Language Long Chain-of-Thought (LCoT), achieved by Reinforcement Learning with Verifiable Rewards (RLVR), has proven effective in enhancing the reasoning capabilities of Large Language Models (LLMs). However, reasoning in current LLMs is primarily generated as plain text, where performing semantic evaluation on such unstructured data creates a computational bottleneck during training. Despite RLVR-based optimization, existing methods still suffer from coarse-grained supervision, reward hacking, high training costs, and poor generalization. To address these issues, we propose the Graph Reasoning Paradigm (GRP), which realizes structured and symbolic reasoning, implemented via graph-structured representations with step-level cognitive labels. Building upon GRP, we further design Process-Aware Stratified Clipping Group Relative Policy Optimization (PASC-GRPO), which leverages structured evaluation to replace semantic evaluation, achieves process-aware verification through graph-structured outcome rewards, and mitigates reward hacking via stratified clipping advantage estimation. Experiments demonstrate significant improvements across mathematical reasoning and code generation tasks. Data, models, and code will be released later. |
| title | Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.12995 |