User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915533354434560 |
|---|---|
| author | Liu, Yuhan Zhang, Michael J. Q. Choi, Eunsol |
| author_facet | Liu, Yuhan Zhang, Michael J. Q. Choi, Eunsol |
| contents | Once language models (LMs) are deployed, they can interact with users long-term, ideally evolving based on their feedback. Asking for direct user feedback can be disruptive; thus, we study harvesting implicit user feedback from user-LM interaction logs. We study two user-LM interaction datasets (WildChat and LMSYS). First, we analyze user feedback in the user-LLM conversation logs, providing insights into when and why such feedback occurs. Second, we study harvesting learning signals from such implicit user feedback. Specifically, we study whether incorporating the contents of user feedback (e.g., user wanted clarification), in addition to the polarity of the feedback, can improve the model performance. We observe mixed results, showing this helps in short human-designed questions (MTBench) but not on longer and more complex questions (WildBench). Together, we provide an in-depth study of implicit user feedback, showing its potential and limitations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_23158 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal Liu, Yuhan Zhang, Michael J. Q. Choi, Eunsol Computation and Language Once language models (LMs) are deployed, they can interact with users long-term, ideally evolving based on their feedback. Asking for direct user feedback can be disruptive; thus, we study harvesting implicit user feedback from user-LM interaction logs. We study two user-LM interaction datasets (WildChat and LMSYS). First, we analyze user feedback in the user-LLM conversation logs, providing insights into when and why such feedback occurs. Second, we study harvesting learning signals from such implicit user feedback. Specifically, we study whether incorporating the contents of user feedback (e.g., user wanted clarification), in addition to the polarity of the feedback, can improve the model performance. We observe mixed results, showing this helps in short human-designed questions (MTBench) but not on longer and more complex questions (WildBench). Together, we provide an in-depth study of implicit user feedback, showing its potential and limitations. |
| title | User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.23158 |