On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915375675867136 |
|---|---|
| author | Chakraborty, Souradip Pourreza, Mohammadreza Sun, Ruoxi Song, Yiwen Scherrer, Nino Huang, Furong Bedi, Amrit Singh Beirami, Ahmad Gu, Jindong Palangi, Hamid Pfister, Tomas |
| author_facet | Chakraborty, Souradip Pourreza, Mohammadreza Sun, Ruoxi Song, Yiwen Scherrer, Nino Huang, Furong Bedi, Amrit Singh Beirami, Ahmad Gu, Jindong Palangi, Hamid Pfister, Tomas |
| contents | Agentic AI workflows (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low. A promising solution is inference-time alignment, which uses extra compute at test time to improve performance. Inference-time alignment relies on three components: sampling, evaluation, and feedback. While most prior work studies sampling and automatic evaluation, feedback remains underexplored. To study the role of feedback, we introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques (reward models or AI-generated textual feedback) between decoding steps. Through IAD, we analyze feedback along four dimensions: (1) its role in the accuracy-compute trade-offs with limited inference budget, (2) quantifying the gains over diversity-only baselines such as best-of-N sampling, (3) effectiveness of composing feedback from reward models versus textual critique, and (4) robustness to noisy or low-quality feedback. Across Sketch2Code, Text2SQL, Intercode, and WebShop, we show that IAD with proper integration of high fidelity feedback leads to consistent gains up to 10 percent absolute performance improvement over various baselines such as best-of-N. Our findings underscore feedback as a crucial knob for inference-time alignment of agentic AI workflows with limited inference budget. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_01931 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows Chakraborty, Souradip Pourreza, Mohammadreza Sun, Ruoxi Song, Yiwen Scherrer, Nino Huang, Furong Bedi, Amrit Singh Beirami, Ahmad Gu, Jindong Palangi, Hamid Pfister, Tomas Computation and Language Agentic AI workflows (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low. A promising solution is inference-time alignment, which uses extra compute at test time to improve performance. Inference-time alignment relies on three components: sampling, evaluation, and feedback. While most prior work studies sampling and automatic evaluation, feedback remains underexplored. To study the role of feedback, we introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques (reward models or AI-generated textual feedback) between decoding steps. Through IAD, we analyze feedback along four dimensions: (1) its role in the accuracy-compute trade-offs with limited inference budget, (2) quantifying the gains over diversity-only baselines such as best-of-N sampling, (3) effectiveness of composing feedback from reward models versus textual critique, and (4) robustness to noisy or low-quality feedback. Across Sketch2Code, Text2SQL, Intercode, and WebShop, we show that IAD with proper integration of high fidelity feedback leads to consistent gains up to 10 percent absolute performance improvement over various baselines such as best-of-N. Our findings underscore feedback as a crucial knob for inference-time alignment of agentic AI workflows with limited inference budget. |
| title | On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2504.01931 |