RLHF and IIA: Perverse Incentives
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929231030648832 |
|---|---|
| author | Xu, Wanqiao Dong, Shi Lu, Xiuyuan Lam, Grace Wen, Zheng Van Roy, Benjamin |
| author_facet | Xu, Wanqiao Dong, Shi Lu, Xiuyuan Lam, Grace Wen, Zheng Van Roy, Benjamin |
| contents | Existing algorithms for reinforcement learning from human feedback (RLHF) can incentivize responses at odds with preferences because they are based on models that assume independence of irrelevant alternatives (IIA). The perverse incentives induced by IIA hinder innovations on query formats and learning algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_01057 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | RLHF and IIA: Perverse Incentives Xu, Wanqiao Dong, Shi Lu, Xiuyuan Lam, Grace Wen, Zheng Van Roy, Benjamin Machine Learning Artificial Intelligence Computation and Language Existing algorithms for reinforcement learning from human feedback (RLHF) can incentivize responses at odds with preferences because they are based on models that assume independence of irrelevant alternatives (IIA). The perverse incentives induced by IIA hinder innovations on query formats and learning algorithms. |
| title | RLHF and IIA: Perverse Incentives |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2312.01057 |