Does Instruction Tuning Make LLMs More Consistent?
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916421129207808 |
|---|---|
| author | Fierro, Constanza Li, Jiaang Søgaard, Anders |
| author_facet | Fierro, Constanza Li, Jiaang Søgaard, Anders |
| contents | The purpose of instruction tuning is enabling zero-shot performance, but instruction tuning has also been shown to improve chain-of-thought reasoning and value alignment (Si et al., 2023). Here we consider the impact on $\textit{consistency}$, i.e., the sensitivity of language models to small perturbations in the input. We compare 10 instruction-tuned LLaMA models to the original LLaMA-7b model and show that almost across-the-board they become more consistent, both in terms of their representations and their predictions in zero-shot and downstream tasks. We explain these improvements through mechanistic analyses of factual recall. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_15206 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Does Instruction Tuning Make LLMs More Consistent? Fierro, Constanza Li, Jiaang Søgaard, Anders Computation and Language The purpose of instruction tuning is enabling zero-shot performance, but instruction tuning has also been shown to improve chain-of-thought reasoning and value alignment (Si et al., 2023). Here we consider the impact on $\textit{consistency}$, i.e., the sensitivity of language models to small perturbations in the input. We compare 10 instruction-tuned LLaMA models to the original LLaMA-7b model and show that almost across-the-board they become more consistent, both in terms of their representations and their predictions in zero-shot and downstream tasks. We explain these improvements through mechanistic analyses of factual recall. |
| title | Does Instruction Tuning Make LLMs More Consistent? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.15206 |