BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908509286694912 |
|---|---|
| author | Wen, Hao Wu, Xinrui Sun, Yi Zhang, Feifei Chen, Liye Wang, Jie Liu, Yunxin Liu, Yunhao Zhang, Ya-Qin Li, Yuanchun |
| author_facet | Wen, Hao Wu, Xinrui Sun, Yi Zhang, Feifei Chen, Liye Wang, Jie Liu, Yunxin Liu, Yunhao Zhang, Ya-Qin Li, Yuanchun |
| contents | Recent advancements in Large Language Models (LLMs) have leveraged increased test-time computation to enhance reasoning capabilities, a strategy that, while effective, incurs significant latency and resource costs, limiting their applicability in real-world time-constrained or cost-sensitive scenarios. This paper introduces BudgetThinker, a novel framework designed to empower LLMs with budget-aware reasoning, enabling precise control over the length of their thought processes. We propose a methodology that periodically inserts special control tokens during inference to continuously inform the model of its remaining token budget. This approach is coupled with a comprehensive two-stage training pipeline, beginning with Supervised Fine-Tuning (SFT) to familiarize the model with budget constraints, followed by a curriculum-based Reinforcement Learning (RL) phase that utilizes a length-aware reward function to optimize for both accuracy and budget adherence. We demonstrate that BudgetThinker significantly surpasses strong baselines in maintaining performance across a variety of reasoning budgets on challenging mathematical benchmarks. Our method provides a scalable and effective solution for developing efficient and controllable LLM reasoning, making advanced models more practical for deployment in resource-constrained and real-time environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17196 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens Wen, Hao Wu, Xinrui Sun, Yi Zhang, Feifei Chen, Liye Wang, Jie Liu, Yunxin Liu, Yunhao Zhang, Ya-Qin Li, Yuanchun Machine Learning Artificial Intelligence Recent advancements in Large Language Models (LLMs) have leveraged increased test-time computation to enhance reasoning capabilities, a strategy that, while effective, incurs significant latency and resource costs, limiting their applicability in real-world time-constrained or cost-sensitive scenarios. This paper introduces BudgetThinker, a novel framework designed to empower LLMs with budget-aware reasoning, enabling precise control over the length of their thought processes. We propose a methodology that periodically inserts special control tokens during inference to continuously inform the model of its remaining token budget. This approach is coupled with a comprehensive two-stage training pipeline, beginning with Supervised Fine-Tuning (SFT) to familiarize the model with budget constraints, followed by a curriculum-based Reinforcement Learning (RL) phase that utilizes a length-aware reward function to optimize for both accuracy and budget adherence. We demonstrate that BudgetThinker significantly surpasses strong baselines in maintaining performance across a variety of reasoning budgets on challenging mathematical benchmarks. Our method provides a scalable and effective solution for developing efficient and controllable LLM reasoning, making advanced models more practical for deployment in resource-constrained and real-time environments. |
| title | BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2508.17196 |