Learning Personalized Agents from Human Feedback
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912911316746240 |
|---|---|
| author | Liang, Kaiqu Kruk, Julia Qian, Shengyi Yang, Xianjun Bi, Shengjie Yao, Yuanshun Nie, Shaoliang Zhang, Mingyang Liu, Lijuan Fisac, Jaime Fernández Zhou, Shuyan Hosseini, Saghar |
| author_facet | Liang, Kaiqu Kruk, Julia Qian, Shengyi Yang, Xianjun Bi, Shengjie Yao, Yuanshun Nie, Shaoliang Zhang, Mingyang Liu, Lijuan Fisac, Jaime Fernández Zhou, Shuyan Hosseini, Saghar |
| contents | Modern AI agents are powerful but often fail to align with the idiosyncratic, evolving preferences of individual users. Prior approaches typically rely on static datasets, either training implicit preference models on interaction history or encoding user profiles in external memory. However, these approaches struggle with new users and with preferences that change over time. We introduce Personalized Agents from Human Feedback (PAHF), a framework for continual personalization in which agents learn online from live interaction using explicit per-user memory. PAHF operationalizes a three-step loop: (1) seeking pre-action clarification to resolve ambiguity, (2) grounding actions in preferences retrieved from memory, and (3) integrating post-action feedback to update memory when preferences drift. To evaluate this capability, we develop a four-phase protocol and two benchmarks in embodied manipulation and online shopping. These benchmarks quantify an agent's ability to learn initial preferences from scratch and subsequently adapt to persona shifts. Our theoretical analysis and empirical results show that integrating explicit memory with dual feedback channels is critical: PAHF learns substantially faster and consistently outperforms both no-memory and single-channel baselines, reducing initial personalization error and enabling rapid adaptation to preference shifts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_16173 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning Personalized Agents from Human Feedback Liang, Kaiqu Kruk, Julia Qian, Shengyi Yang, Xianjun Bi, Shengjie Yao, Yuanshun Nie, Shaoliang Zhang, Mingyang Liu, Lijuan Fisac, Jaime Fernández Zhou, Shuyan Hosseini, Saghar Artificial Intelligence Computation and Language Machine Learning Modern AI agents are powerful but often fail to align with the idiosyncratic, evolving preferences of individual users. Prior approaches typically rely on static datasets, either training implicit preference models on interaction history or encoding user profiles in external memory. However, these approaches struggle with new users and with preferences that change over time. We introduce Personalized Agents from Human Feedback (PAHF), a framework for continual personalization in which agents learn online from live interaction using explicit per-user memory. PAHF operationalizes a three-step loop: (1) seeking pre-action clarification to resolve ambiguity, (2) grounding actions in preferences retrieved from memory, and (3) integrating post-action feedback to update memory when preferences drift. To evaluate this capability, we develop a four-phase protocol and two benchmarks in embodied manipulation and online shopping. These benchmarks quantify an agent's ability to learn initial preferences from scratch and subsequently adapt to persona shifts. Our theoretical analysis and empirical results show that integrating explicit memory with dual feedback channels is critical: PAHF learns substantially faster and consistently outperforms both no-memory and single-channel baselines, reducing initial personalization error and enabling rapid adaptation to preference shifts. |
| title | Learning Personalized Agents from Human Feedback |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2602.16173 |