Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.06137 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908396553240576 |
|---|---|
| author | Jin, Rihui Xin, Zheyu Xie, Xing Li, Zuoyi Qi, Guilin Chen, Yongrui Dai, Xinbang Wu, Tongtong Haffari, Gholamreza |
| author_facet | Jin, Rihui Xin, Zheyu Xie, Xing Li, Zuoyi Qi, Guilin Chen, Yongrui Dai, Xinbang Wu, Tongtong Haffari, Gholamreza |
| contents | Table reasoning (TR) requires structured reasoning over semi-structured tabular data and remains challenging, particularly for small language models (SLMs, e.g., LLaMA-8B) due to their limited capacity compared to large LMs (LLMs, e.g., GPT-4o). To narrow this gap, we explore program-based TR (P-TR), which circumvents key limitations of text-based TR (T-TR), notably in numerical reasoning, by generating executable programs. However, applying P-TR to SLMs introduces two challenges: (i) vulnerability to heterogeneity in table layouts, and (ii) inconsistency in reasoning due to limited code generation capability. We propose Table-r1, a two-stage P-TR method designed for SLMs. Stage 1 introduces an innovative self-supervised learning task, Layout Transformation Inference, to improve tabular layout generalization from a programmatic view. Stage 2 adopts a mix-paradigm variant of Group Relative Policy Optimization, enhancing P-TR consistency while allowing dynamic fallback to T-TR when needed. Experiments on four TR benchmarks demonstrate that Table-r1 outperforms all SLM-based methods, achieving at least a 15% accuracy improvement over the base model (LLaMA-8B) across all datasets and reaching performance competitive with LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_06137 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models Jin, Rihui Xin, Zheyu Xie, Xing Li, Zuoyi Qi, Guilin Chen, Yongrui Dai, Xinbang Wu, Tongtong Haffari, Gholamreza Machine Learning Computation and Language Table reasoning (TR) requires structured reasoning over semi-structured tabular data and remains challenging, particularly for small language models (SLMs, e.g., LLaMA-8B) due to their limited capacity compared to large LMs (LLMs, e.g., GPT-4o). To narrow this gap, we explore program-based TR (P-TR), which circumvents key limitations of text-based TR (T-TR), notably in numerical reasoning, by generating executable programs. However, applying P-TR to SLMs introduces two challenges: (i) vulnerability to heterogeneity in table layouts, and (ii) inconsistency in reasoning due to limited code generation capability. We propose Table-r1, a two-stage P-TR method designed for SLMs. Stage 1 introduces an innovative self-supervised learning task, Layout Transformation Inference, to improve tabular layout generalization from a programmatic view. Stage 2 adopts a mix-paradigm variant of Group Relative Policy Optimization, enhancing P-TR consistency while allowing dynamic fallback to T-TR when needed. Experiments on four TR benchmarks demonstrate that Table-r1 outperforms all SLM-based methods, achieving at least a 15% accuracy improvement over the base model (LLaMA-8B) across all datasets and reaching performance competitive with LLMs. |
| title | Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2506.06137 |