Enhancing Preference-based Linear Bandits via Human Response Time
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909447029260288 |
|---|---|
| author | Li, Shen Zhang, Yuyang Ren, Zhaolin Liang, Claire Li, Na Shah, Julie A. |
| author_facet | Li, Shen Zhang, Yuyang Ren, Zhaolin Liang, Claire Li, Na Shah, Julie A. |
| contents | Interactive preference learning systems infer human preferences by presenting queries as pairs of options and collecting binary choices. Although binary choices are simple and widely used, they provide limited information about preference strength. To address this, we leverage human response times, which are inversely related to preference strength, as an additional signal. We propose a computationally efficient method that combines choices and response times to estimate human utility functions, grounded in the EZ diffusion model from psychology. Theoretical and empirical analyses show that for queries with strong preferences, response times complement choices by providing extra information about preference strength, leading to significantly improved utility estimation. We incorporate this estimator into preference-based linear bandits for fixed-budget best-arm identification. Simulations on three real-world datasets demonstrate that using response times significantly accelerates preference learning compared to choice-only approaches. Additional materials, such as code, slides, and talk video, are available at https://shenlirobot.github.io/pages/NeurIPS24.html |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_05798 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Preference-based Linear Bandits via Human Response Time Li, Shen Zhang, Yuyang Ren, Zhaolin Liang, Claire Li, Na Shah, Julie A. Machine Learning Artificial Intelligence Human-Computer Interaction Econometrics Interactive preference learning systems infer human preferences by presenting queries as pairs of options and collecting binary choices. Although binary choices are simple and widely used, they provide limited information about preference strength. To address this, we leverage human response times, which are inversely related to preference strength, as an additional signal. We propose a computationally efficient method that combines choices and response times to estimate human utility functions, grounded in the EZ diffusion model from psychology. Theoretical and empirical analyses show that for queries with strong preferences, response times complement choices by providing extra information about preference strength, leading to significantly improved utility estimation. We incorporate this estimator into preference-based linear bandits for fixed-budget best-arm identification. Simulations on three real-world datasets demonstrate that using response times significantly accelerates preference learning compared to choice-only approaches. Additional materials, such as code, slides, and talk video, are available at https://shenlirobot.github.io/pages/NeurIPS24.html |
| title | Enhancing Preference-based Linear Bandits via Human Response Time |
| topic | Machine Learning Artificial Intelligence Human-Computer Interaction Econometrics |
| url | https://arxiv.org/abs/2409.05798 |