ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908999255851008 |
|---|---|
| author | Lin, Jianghao Shi, Yuanyuan Peng, Xin Ding, Renjie Wang, Hairui Peng, Yuxuan Bai, Bizhe Song, Weixi Bai, Fengshuo Chai, Huacan Zhang, Weinan Huang, Fei Wen, Ying |
| author_facet | Lin, Jianghao Shi, Yuanyuan Peng, Xin Ding, Renjie Wang, Hairui Peng, Yuxuan Bai, Bizhe Song, Weixi Bai, Fengshuo Chai, Huacan Zhang, Weinan Huang, Fei Wen, Ying |
| contents | Large language models (LLMs) excel at function calling, but inference scaling has been explored mainly for unstructured generation. We propose an inference-scaling framework for structured outputs that combines fine-grained beam search with \textbf{ToolPRM}, a process reward model scoring each intra-call decision (function name and argument filling). We build the first fine-grained intra-call supervision dataset via function masking, rollout collection, and step-level annotation. ToolPRM outperforms outcome and coarse-grained reward models in predictive accuracy and yields consistent test-time gains on multiple function-calling benchmarks. We further show that structured generation follows ``\textbf{explore more but retain less}'', since early JSON errors are unrecoverable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14703 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling Lin, Jianghao Shi, Yuanyuan Peng, Xin Ding, Renjie Wang, Hairui Peng, Yuxuan Bai, Bizhe Song, Weixi Bai, Fengshuo Chai, Huacan Zhang, Weinan Huang, Fei Wen, Ying Artificial Intelligence Large language models (LLMs) excel at function calling, but inference scaling has been explored mainly for unstructured generation. We propose an inference-scaling framework for structured outputs that combines fine-grained beam search with \textbf{ToolPRM}, a process reward model scoring each intra-call decision (function name and argument filling). We build the first fine-grained intra-call supervision dataset via function masking, rollout collection, and step-level annotation. ToolPRM outperforms outcome and coarse-grained reward models in predictive accuracy and yields consistent test-time gains on multiple function-calling benchmarks. We further show that structured generation follows ``\textbf{explore more but retain less}'', since early JSON errors are unrecoverable. |
| title | ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.14703 |