Ada-RS: Adaptive Rejection Sampling for Selective Thinking
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911462819102720 |
|---|---|
| author | Ge, Yirou Li, Yixi Chiu, Alec Shekhar, Shivani Pan, Zijie Thangali, Avinash Chuang, Yun-Shiuan Kulkarni, Chaitanya Kona, Uma Pang, Linsey Mehrotra, Prakhar |
| author_facet | Ge, Yirou Li, Yixi Chiu, Alec Shekhar, Shivani Pan, Zijie Thangali, Avinash Chuang, Yun-Shiuan Kulkarni, Chaitanya Kona, Uma Pang, Linsey Mehrotra, Prakhar |
| contents | Large language models (LLMs) are increasingly being deployed in cost and latency-sensitive settings. While chain-of-thought improves reasoning, it can waste tokens on simple requests. We study selective thinking for tool-using LLMs and introduce Adaptive Rejection Sampling (Ada-RS), an algorithm-agnostic sample filtering framework for learning selective and efficient reasoning. For each given context, Ada-RS scores multiple sampled completions with an adaptive length-penalized reward then applies stochastic rejection sampling to retain only high-reward candidates (or preference pairs) for downstream optimization. We demonstrate how Ada-RS plugs into both preference pair (e.g. DPO) or grouped policy optimization strategies (e.g. DAPO). Using Qwen3-8B with LoRA on a synthetic tool call-oriented e-commerce benchmark, Ada-RS improves the accuracy-efficiency frontier over standard algorithms by reducing average output tokens by up to 80% and reducing thinking rate by up to 95% while maintaining or improving tool call accuracy. These results highlight that training-signal selection is a powerful lever for efficient reasoning in latency-sensitive deployments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_19519 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Ada-RS: Adaptive Rejection Sampling for Selective Thinking Ge, Yirou Li, Yixi Chiu, Alec Shekhar, Shivani Pan, Zijie Thangali, Avinash Chuang, Yun-Shiuan Kulkarni, Chaitanya Kona, Uma Pang, Linsey Mehrotra, Prakhar Artificial Intelligence Machine Learning Large language models (LLMs) are increasingly being deployed in cost and latency-sensitive settings. While chain-of-thought improves reasoning, it can waste tokens on simple requests. We study selective thinking for tool-using LLMs and introduce Adaptive Rejection Sampling (Ada-RS), an algorithm-agnostic sample filtering framework for learning selective and efficient reasoning. For each given context, Ada-RS scores multiple sampled completions with an adaptive length-penalized reward then applies stochastic rejection sampling to retain only high-reward candidates (or preference pairs) for downstream optimization. We demonstrate how Ada-RS plugs into both preference pair (e.g. DPO) or grouped policy optimization strategies (e.g. DAPO). Using Qwen3-8B with LoRA on a synthetic tool call-oriented e-commerce benchmark, Ada-RS improves the accuracy-efficiency frontier over standard algorithms by reducing average output tokens by up to 80% and reducing thinking rate by up to 95% while maintaining or improving tool call accuracy. These results highlight that training-signal selection is a powerful lever for efficient reasoning in latency-sensitive deployments. |
| title | Ada-RS: Adaptive Rejection Sampling for Selective Thinking |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2602.19519 |