CollabLLM: From Passive Responders to Active Collaborators
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912509966942208 |
|---|---|
| author | Wu, Shirley Galley, Michel Peng, Baolin Cheng, Hao Li, Gavin Dou, Yao Cai, Weixin Zou, James Leskovec, Jure Gao, Jianfeng |
| author_facet | Wu, Shirley Galley, Michel Peng, Baolin Cheng, Hao Li, Gavin Dou, Yao Cai, Weixin Zou, James Leskovec, Jure Gao, Jianfeng |
| contents | Large Language Models are typically trained with next-turn rewards, limiting their ability to optimize for long-term interaction. As a result, they often respond passively to ambiguous or open-ended user requests, failing to help users reach their ultimate intents and leading to inefficient conversations. To address these limitations, we introduce CollabLLM, a novel and general training framework that enhances multiturn human-LLM collaboration. Its key innovation is a collaborative simulation that estimates the long-term contribution of responses using Multiturn-aware Rewards. By reinforcement fine-tuning these rewards, CollabLLM goes beyond responding to user requests, and actively uncovers user intent and offers insightful suggestions-a key step towards more human-centered AI. We also devise a multiturn interaction benchmark with three challenging tasks such as document creation. CollabLLM significantly outperforms our baselines with averages of 18.5% higher task performance and 46.3% improved interactivity by LLM judges. Finally, we conduct a large user study with 201 judges, where CollabLLM increases user satisfaction by 17.6% and reduces user spent time by 10.4%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_00640 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CollabLLM: From Passive Responders to Active Collaborators Wu, Shirley Galley, Michel Peng, Baolin Cheng, Hao Li, Gavin Dou, Yao Cai, Weixin Zou, James Leskovec, Jure Gao, Jianfeng Artificial Intelligence Large Language Models are typically trained with next-turn rewards, limiting their ability to optimize for long-term interaction. As a result, they often respond passively to ambiguous or open-ended user requests, failing to help users reach their ultimate intents and leading to inefficient conversations. To address these limitations, we introduce CollabLLM, a novel and general training framework that enhances multiturn human-LLM collaboration. Its key innovation is a collaborative simulation that estimates the long-term contribution of responses using Multiturn-aware Rewards. By reinforcement fine-tuning these rewards, CollabLLM goes beyond responding to user requests, and actively uncovers user intent and offers insightful suggestions-a key step towards more human-centered AI. We also devise a multiturn interaction benchmark with three challenging tasks such as document creation. CollabLLM significantly outperforms our baselines with averages of 18.5% higher task performance and 46.3% improved interactivity by LLM judges. Finally, we conduct a large user study with 201 judges, where CollabLLM increases user satisfaction by 17.6% and reduces user spent time by 10.4%. |
| title | CollabLLM: From Passive Responders to Active Collaborators |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2502.00640 |