Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915335093878784 |
|---|---|
| author | Shen, Junhong Bai, Hao Zhang, Lunjun Zhou, Yifei Setlur, Amrith Tong, Shengbang Caples, Diego Jiang, Nan Zhang, Tong Talwalkar, Ameet Kumar, Aviral |
| author_facet | Shen, Junhong Bai, Hao Zhang, Lunjun Zhou, Yifei Setlur, Amrith Tong, Shengbang Caples, Diego Jiang, Nan Zhang, Tong Talwalkar, Ameet Kumar, Aviral |
| contents | The current paradigm of test-time scaling relies on generating long reasoning traces ("thinking" more) before producing a response. In agent problems that require interaction, this can be done by generating thinking traces before acting in the world. However, this process does not allow agents to acquire new information from the environment or adapt their behavior over time. In this work, we propose to scale test-time interaction, an untapped dimension of test-time scaling that increases the agent's interaction horizon to enable running rich behaviors such as exploration, backtracking, and dynamic re-planning within a single rollout. To demonstrate the promise of this scaling dimension, we study the domain of web agents. We first show that even prompting-based interaction scaling without any training can improve task success on web benchmarks non-trivially. Building on this, we introduce TTI (Test-Time Interaction), a curriculum-based online reinforcement learning (RL) approach that trains agents by adaptively adjusting their rollout lengths. Using a Gemma 3 12B model, TTI produces state-of-the-art open-source, open-data web agents on WebVoyager and WebArena benchmarks. We further show that TTI enables agents to balance exploration and exploitation adaptively. Our results establish interaction scaling as a powerful, complementary axis to scaling per-step compute, offering new avenues for training adaptive agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_07976 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction Shen, Junhong Bai, Hao Zhang, Lunjun Zhou, Yifei Setlur, Amrith Tong, Shengbang Caples, Diego Jiang, Nan Zhang, Tong Talwalkar, Ameet Kumar, Aviral Machine Learning Artificial Intelligence The current paradigm of test-time scaling relies on generating long reasoning traces ("thinking" more) before producing a response. In agent problems that require interaction, this can be done by generating thinking traces before acting in the world. However, this process does not allow agents to acquire new information from the environment or adapt their behavior over time. In this work, we propose to scale test-time interaction, an untapped dimension of test-time scaling that increases the agent's interaction horizon to enable running rich behaviors such as exploration, backtracking, and dynamic re-planning within a single rollout. To demonstrate the promise of this scaling dimension, we study the domain of web agents. We first show that even prompting-based interaction scaling without any training can improve task success on web benchmarks non-trivially. Building on this, we introduce TTI (Test-Time Interaction), a curriculum-based online reinforcement learning (RL) approach that trains agents by adaptively adjusting their rollout lengths. Using a Gemma 3 12B model, TTI produces state-of-the-art open-source, open-data web agents on WebVoyager and WebArena benchmarks. We further show that TTI enables agents to balance exploration and exploitation adaptively. Our results establish interaction scaling as a powerful, complementary axis to scaling per-step compute, offering new avenues for training adaptive agents. |
| title | Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2506.07976 |