Understanding The Effect Of Temperature On Alignment With Human Opinions
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916482315714560 |
|---|---|
| author | Pavlovic, Maja Poesio, Massimo |
| author_facet | Pavlovic, Maja Poesio, Massimo |
| contents | With the increasing capabilities of LLMs, recent studies focus on understanding whose opinions are represented by them and how to effectively extract aligned opinion distributions. We conducted an empirical analysis of three straightforward methods for obtaining distributions and evaluated the results across a variety of metrics. Our findings suggest that sampling and log-probability approaches with simple parameter adjustments can return better aligned outputs in subjective tasks compared to direct prompting. Yet, assuming models reflect human opinions may be limiting, highlighting the need for further research on how human subjectivity affects model uncertainty. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_10080 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding The Effect Of Temperature On Alignment With Human Opinions Pavlovic, Maja Poesio, Massimo Computation and Language Computers and Society With the increasing capabilities of LLMs, recent studies focus on understanding whose opinions are represented by them and how to effectively extract aligned opinion distributions. We conducted an empirical analysis of three straightforward methods for obtaining distributions and evaluated the results across a variety of metrics. Our findings suggest that sampling and log-probability approaches with simple parameter adjustments can return better aligned outputs in subjective tasks compared to direct prompting. Yet, assuming models reflect human opinions may be limiting, highlighting the need for further research on how human subjectivity affects model uncertainty. |
| title | Understanding The Effect Of Temperature On Alignment With Human Opinions |
| topic | Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2411.10080 |