Too Big to Fool: Resisting Deception in Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910745066733568 |
|---|---|
| author | Samsami, Mohammad Reza Richter, Mats Leon Rodriguez, Juan Thakkar, Megh Chandar, Sarath Gasse, Maxime |
| author_facet | Samsami, Mohammad Reza Richter, Mats Leon Rodriguez, Juan Thakkar, Megh Chandar, Sarath Gasse, Maxime |
| contents | Large language models must balance their weight-encoded knowledge with in-context information from prompts to generate accurate responses. This paper investigates this interplay by analyzing how models of varying capacities within the same family handle intentionally misleading in-context information. Our experiments demonstrate that larger models exhibit higher resilience to deceptive prompts, showcasing an advanced ability to interpret and integrate prompt information with their internal knowledge. Furthermore, we find that larger models outperform smaller ones in following legitimate instructions, indicating that their resilience is not due to disregarding in-context information. We also show that this phenomenon is likely not a result of memorization but stems from the models' ability to better leverage implicit task-relevant information from the prompt alongside their internally stored knowledge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_10558 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Too Big to Fool: Resisting Deception in Language Models Samsami, Mohammad Reza Richter, Mats Leon Rodriguez, Juan Thakkar, Megh Chandar, Sarath Gasse, Maxime Computation and Language Artificial Intelligence Machine Learning Large language models must balance their weight-encoded knowledge with in-context information from prompts to generate accurate responses. This paper investigates this interplay by analyzing how models of varying capacities within the same family handle intentionally misleading in-context information. Our experiments demonstrate that larger models exhibit higher resilience to deceptive prompts, showcasing an advanced ability to interpret and integrate prompt information with their internal knowledge. Furthermore, we find that larger models outperform smaller ones in following legitimate instructions, indicating that their resilience is not due to disregarding in-context information. We also show that this phenomenon is likely not a result of memorization but stems from the models' ability to better leverage implicit task-relevant information from the prompt alongside their internally stored knowledge. |
| title | Too Big to Fool: Resisting Deception in Language Models |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2412.10558 |