Optimal Design for Human Preference Elicitation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911448717852672 |
|---|---|
| author | Mukherjee, Subhojyoti Lalitha, Anusha Kalantari, Kousha Deshmukh, Aniket Liu, Ge Ma, Yifei Kveton, Branislav |
| author_facet | Mukherjee, Subhojyoti Lalitha, Anusha Kalantari, Kousha Deshmukh, Aniket Liu, Ge Ma, Yifei Kveton, Branislav |
| contents | Learning of preference models from human feedback has been central to recent advances in artificial intelligence. Motivated by the cost of obtaining high-quality human annotations, we study efficient human preference elicitation for learning preference models. The key idea in our work is to generalize optimal designs, an approach to computing optimal information-gathering policies, to lists of items that represent potential questions with answers. The policy is a distribution over the lists and we elicit preferences from them proportionally to their probabilities. To show the generality of our ideas, we study both absolute and ranking feedback models on items in the list. We design efficient algorithms for both and analyze them. Finally, we demonstrate that our algorithms are practical by evaluating them on existing question-answering problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_13895 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Optimal Design for Human Preference Elicitation Mukherjee, Subhojyoti Lalitha, Anusha Kalantari, Kousha Deshmukh, Aniket Liu, Ge Ma, Yifei Kveton, Branislav Machine Learning Learning of preference models from human feedback has been central to recent advances in artificial intelligence. Motivated by the cost of obtaining high-quality human annotations, we study efficient human preference elicitation for learning preference models. The key idea in our work is to generalize optimal designs, an approach to computing optimal information-gathering policies, to lists of items that represent potential questions with answers. The policy is a distribution over the lists and we elicit preferences from them proportionally to their probabilities. To show the generality of our ideas, we study both absolute and ranking feedback models on items in the list. We design efficient algorithms for both and analyze them. Finally, we demonstrate that our algorithms are practical by evaluating them on existing question-answering problems. |
| title | Optimal Design for Human Preference Elicitation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2404.13895 |