ADAPT: Actively Discovering and Adapting to Preferences for any Task
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915229508567040 |
|---|---|
| author | Patel, Maithili Puig, Xavier Desai, Ruta Mottaghi, Roozbeh Chernova, Sonia Truong, Joanne Rai, Akshara |
| author_facet | Patel, Maithili Puig, Xavier Desai, Ruta Mottaghi, Roozbeh Chernova, Sonia Truong, Joanne Rai, Akshara |
| contents | Assistive agents should be able to perform under-specified long-horizon tasks while respecting user preferences. We introduce Actively Discovering and Adapting to Preferences for any Task (ADAPT) -- a benchmark designed to evaluate agents' ability to adhere to user preferences across various household tasks through active questioning. Next, we propose Reflection-DPO, a novel training approach for adapting large language models (LLMs) to the task of active questioning. Reflection-DPO finetunes a 'student' LLM to follow the actions of a privileged 'teacher' LLM, and optionally ask a question to gather necessary information to better predict the teacher action. We find that prior approaches that use state-of-the-art LLMs fail to sufficiently follow user preferences in ADAPT due to insufficient questioning and poor adherence to elicited preferences. In contrast, Reflection-DPO achieves a higher rate of satisfying user preferences, outperforming a zero-shot chain-of-thought baseline by 6.1% on unseen users. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04040 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ADAPT: Actively Discovering and Adapting to Preferences for any Task Patel, Maithili Puig, Xavier Desai, Ruta Mottaghi, Roozbeh Chernova, Sonia Truong, Joanne Rai, Akshara Artificial Intelligence Robotics Assistive agents should be able to perform under-specified long-horizon tasks while respecting user preferences. We introduce Actively Discovering and Adapting to Preferences for any Task (ADAPT) -- a benchmark designed to evaluate agents' ability to adhere to user preferences across various household tasks through active questioning. Next, we propose Reflection-DPO, a novel training approach for adapting large language models (LLMs) to the task of active questioning. Reflection-DPO finetunes a 'student' LLM to follow the actions of a privileged 'teacher' LLM, and optionally ask a question to gather necessary information to better predict the teacher action. We find that prior approaches that use state-of-the-art LLMs fail to sufficiently follow user preferences in ADAPT due to insufficient questioning and poor adherence to elicited preferences. In contrast, Reflection-DPO achieves a higher rate of satisfying user preferences, outperforming a zero-shot chain-of-thought baseline by 6.1% on unseen users. |
| title | ADAPT: Actively Discovering and Adapting to Preferences for any Task |
| topic | Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2504.04040 |