Implicit meta-learning may lead language models to trust more reliable sources
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909252715544576 |
|---|---|
| author | Krasheninnikov, Dmitrii Krasheninnikov, Egor Mlodozeniec, Bruno Maharaj, Tegan Krueger, David |
| author_facet | Krasheninnikov, Dmitrii Krasheninnikov, Egor Mlodozeniec, Bruno Maharaj, Tegan Krueger, David |
| contents | We demonstrate that LLMs may learn indicators of document usefulness and modulate their updates accordingly. We introduce random strings ("tags") as indicators of usefulness in a synthetic fine-tuning dataset. Fine-tuning on this dataset leads to implicit meta-learning (IML): in further fine-tuning, the model updates to make more use of text that is tagged as useful. We perform a thorough empirical investigation of this phenomenon, finding (among other things) that (i) it occurs in both pretrained LLMs and those trained from scratch, as well as on a vision task, and (ii) larger models and smaller batch sizes tend to give more IML. We also use probing to examine how IML changes the way models store knowledge in their parameters. Finally, we reflect on what our results might imply about capabilities, risks, and controllability of future AI systems. Our code can be found at https://github.com/krasheninnikov/internalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_15047 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Implicit meta-learning may lead language models to trust more reliable sources Krasheninnikov, Dmitrii Krasheninnikov, Egor Mlodozeniec, Bruno Maharaj, Tegan Krueger, David Machine Learning Artificial Intelligence We demonstrate that LLMs may learn indicators of document usefulness and modulate their updates accordingly. We introduce random strings ("tags") as indicators of usefulness in a synthetic fine-tuning dataset. Fine-tuning on this dataset leads to implicit meta-learning (IML): in further fine-tuning, the model updates to make more use of text that is tagged as useful. We perform a thorough empirical investigation of this phenomenon, finding (among other things) that (i) it occurs in both pretrained LLMs and those trained from scratch, as well as on a vision task, and (ii) larger models and smaller batch sizes tend to give more IML. We also use probing to examine how IML changes the way models store knowledge in their parameters. Finally, we reflect on what our results might imply about capabilities, risks, and controllability of future AI systems. Our code can be found at https://github.com/krasheninnikov/internalization. |
| title | Implicit meta-learning may lead language models to trust more reliable sources |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2310.15047 |