Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912062012129280 |
|---|---|
| author | Li, Jiahuan Cao, Yiqing Huang, Shujian Chen, Jiajun |
| author_facet | Li, Jiahuan Cao, Yiqing Huang, Shujian Chen, Jiajun |
| contents | Having been trained on massive pretraining data, large language models have shown excellent performance on many knowledge-intensive tasks. However, pretraining data tends to contain misleading and even conflicting information, and it is intriguing to understand how LLMs handle these noisy data during training. In this study, we systematically analyze LLMs' learning preferences for data with conflicting knowledge. We find that pretrained LLMs establish learning preferences similar to humans, i.e., preferences towards formal texts and texts with fewer spelling errors, resulting in faster learning and more favorable treatment of knowledge in data with such features when facing conflicts. This finding is generalizable across models and languages and is more evident in larger models. An in-depth analysis reveals that LLMs tend to trust data with features that signify consistency with the majority of data, and it is possible to instill new preferences and erase old ones by manipulating the degree of consistency with the majority data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_04784 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge Li, Jiahuan Cao, Yiqing Huang, Shujian Chen, Jiajun Computation and Language Having been trained on massive pretraining data, large language models have shown excellent performance on many knowledge-intensive tasks. However, pretraining data tends to contain misleading and even conflicting information, and it is intriguing to understand how LLMs handle these noisy data during training. In this study, we systematically analyze LLMs' learning preferences for data with conflicting knowledge. We find that pretrained LLMs establish learning preferences similar to humans, i.e., preferences towards formal texts and texts with fewer spelling errors, resulting in faster learning and more favorable treatment of knowledge in data with such features when facing conflicts. This finding is generalizable across models and languages and is more evident in larger models. An in-depth analysis reveals that LLMs tend to trust data with features that signify consistency with the majority of data, and it is possible to instill new preferences and erase old ones by manipulating the degree of consistency with the majority data. |
| title | Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.04784 |