Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2510.25744 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909877891235840 |
|---|---|
| author | Shen, Shannon Zejiang Chen, Valerie Gu, Ken Ross, Alexis Ma, Zixian Ross, Jillian Gu, Alex Si, Chenglei Chi, Wayne Peng, Andi Shen, Jocelyn J Talwalkar, Ameet Wu, Tongshuang Sontag, David |
| author_facet | Shen, Shannon Zejiang Chen, Valerie Gu, Ken Ross, Alexis Ma, Zixian Ross, Jillian Gu, Alex Si, Chenglei Chi, Wayne Peng, Andi Shen, Jocelyn J Talwalkar, Ameet Wu, Tongshuang Sontag, David |
| contents | Current evaluations of agents remain centered around one-shot task completion, failing to account for the inherently iterative and collaborative nature of many real-world problems, where human goals are often underspecified and evolve. We argue for a shift from building and assessing task completion agents to developing collaborative agents, assessed not only by the quality of their final outputs but by how well they engage with and enhance human effort throughout the problem-solving process. To support this shift, we introduce collaborative effort scaling, a framework that captures how an agent's utility grows with increasing user involvement. Through case studies and simulated evaluations, we show that state-of-the-art agents often underperform in multi-turn, real-world scenarios, revealing a missing ingredient in agent design: the ability to sustain engagement and scaffold user understanding. Collaborative effort scaling offers a lens for diagnosing agent behavior and guiding development toward more effective interactions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_25744 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Completion $\neq$ Collaboration: Scaling Collaborative Effort with Agents Shen, Shannon Zejiang Chen, Valerie Gu, Ken Ross, Alexis Ma, Zixian Ross, Jillian Gu, Alex Si, Chenglei Chi, Wayne Peng, Andi Shen, Jocelyn J Talwalkar, Ameet Wu, Tongshuang Sontag, David Computation and Language Artificial Intelligence Current evaluations of agents remain centered around one-shot task completion, failing to account for the inherently iterative and collaborative nature of many real-world problems, where human goals are often underspecified and evolve. We argue for a shift from building and assessing task completion agents to developing collaborative agents, assessed not only by the quality of their final outputs but by how well they engage with and enhance human effort throughout the problem-solving process. To support this shift, we introduce collaborative effort scaling, a framework that captures how an agent's utility grows with increasing user involvement. Through case studies and simulated evaluations, we show that state-of-the-art agents often underperform in multi-turn, real-world scenarios, revealing a missing ingredient in agent design: the ability to sustain engagement and scaffold user understanding. Collaborative effort scaling offers a lens for diagnosing agent behavior and guiding development toward more effective interactions. |
| title | Completion $\neq$ Collaboration: Scaling Collaborative Effort with Agents |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.25744 |