Role-Play Paradox in Large Language Models: Reasoning Performance Gains and Ethical Dilemmas
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913675194925056 |
|---|---|
| author | Zhao, Jinman Qian, Zifan Cao, Linbo Wang, Yining Ding, Yitian Hu, Yulan Zhang, Zeyu Jin, Zeyong |
| author_facet | Zhao, Jinman Qian, Zifan Cao, Linbo Wang, Yining Ding, Yitian Hu, Yulan Zhang, Zeyu Jin, Zeyong |
| contents | Role-play in large language models (LLMs) enhances their ability to generate contextually relevant and high-quality responses by simulating diverse cognitive perspectives. However, our study identifies significant risks associated with this technique. First, we demonstrate that autotuning, a method used to auto-select models' roles based on the question, can lead to the generation of harmful outputs, even when the model is tasked with adopting neutral roles. Second, we investigate how different roles affect the likelihood of generating biased or harmful content. Through testing on benchmarks containing stereotypical and harmful questions, we find that role-play consistently amplifies the risk of biased outputs. Our results underscore the need for careful consideration of both role simulation and tuning processes when deploying LLMs in sensitive or high-stakes contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13979 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Role-Play Paradox in Large Language Models: Reasoning Performance Gains and Ethical Dilemmas Zhao, Jinman Qian, Zifan Cao, Linbo Wang, Yining Ding, Yitian Hu, Yulan Zhang, Zeyu Jin, Zeyong Computation and Language Role-play in large language models (LLMs) enhances their ability to generate contextually relevant and high-quality responses by simulating diverse cognitive perspectives. However, our study identifies significant risks associated with this technique. First, we demonstrate that autotuning, a method used to auto-select models' roles based on the question, can lead to the generation of harmful outputs, even when the model is tasked with adopting neutral roles. Second, we investigate how different roles affect the likelihood of generating biased or harmful content. Through testing on benchmarks containing stereotypical and harmful questions, we find that role-play consistently amplifies the risk of biased outputs. Our results underscore the need for careful consideration of both role simulation and tuning processes when deploying LLMs in sensitive or high-stakes contexts. |
| title | Role-Play Paradox in Large Language Models: Reasoning Performance Gains and Ethical Dilemmas |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.13979 |