Scaling Evaluation-time Compute with Reasoning Models as Evaluators
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914579693436928 |
|---|---|
| author | Kim, Seungone Wu, Ian Lee, Jinu Yue, Xiang Lee, Seongyun Moon, Mingyeong Lawrence, Carolin Gashteovski, Kiril Hockenmaier, Julia Neubig, Graham Welleck, Sean |
| author_facet | Kim, Seungone Wu, Ian Lee, Jinu Yue, Xiang Lee, Seongyun Moon, Mingyeong Lawrence, Carolin Gashteovski, Kiril Hockenmaier, Julia Neubig, Graham Welleck, Sean |
| contents | As language model (LM) outputs get more and more natural, it is becoming more difficult than ever to evaluate their quality. Simultaneously, increasing LMs' "thinking" time through scaling test-time compute has proven an effective technique to solve challenging problems in domains such as math and code. This raises a natural question: can an LM's evaluation capability also be improved by spending more test-time compute? To answer this, we investigate employing reasoning models-LMs that natively generate long chain-of-thought reasoning-as evaluators. Specifically, we examine methods to leverage more test-time compute by (1) using reasoning models, and (2) prompting these models to evaluate not only the response as a whole (i.e., outcome evaluation) but also assess each step in the response separately (i.e., process evaluation). In experiments, we observe that the evaluator's performance improves monotonically when generating more reasoning tokens, similar to the trends observed in LM-based generation. Furthermore, we use these more accurate evaluators to rerank multiple generations, and demonstrate that spending more compute at evaluation time can be as effective as using more compute at generation time in improving an LM's problem-solving capability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_19877 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scaling Evaluation-time Compute with Reasoning Models as Evaluators Kim, Seungone Wu, Ian Lee, Jinu Yue, Xiang Lee, Seongyun Moon, Mingyeong Lawrence, Carolin Gashteovski, Kiril Hockenmaier, Julia Neubig, Graham Welleck, Sean Computation and Language As language model (LM) outputs get more and more natural, it is becoming more difficult than ever to evaluate their quality. Simultaneously, increasing LMs' "thinking" time through scaling test-time compute has proven an effective technique to solve challenging problems in domains such as math and code. This raises a natural question: can an LM's evaluation capability also be improved by spending more test-time compute? To answer this, we investigate employing reasoning models-LMs that natively generate long chain-of-thought reasoning-as evaluators. Specifically, we examine methods to leverage more test-time compute by (1) using reasoning models, and (2) prompting these models to evaluate not only the response as a whole (i.e., outcome evaluation) but also assess each step in the response separately (i.e., process evaluation). In experiments, we observe that the evaluator's performance improves monotonically when generating more reasoning tokens, similar to the trends observed in LM-based generation. Furthermore, we use these more accurate evaluators to rerank multiple generations, and demonstrate that spending more compute at evaluation time can be as effective as using more compute at generation time in improving an LM's problem-solving capability. |
| title | Scaling Evaluation-time Compute with Reasoning Models as Evaluators |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.19877 |