Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917917196550144 |
|---|---|
| author | Zhao, Yu Devoto, Alessio Hong, Giwon Du, Xiaotang Gema, Aryo Pradipta Wang, Hongru He, Xuanli Wong, Kam-Fai Minervini, Pasquale |
| author_facet | Zhao, Yu Devoto, Alessio Hong, Giwon Du, Xiaotang Gema, Aryo Pradipta Wang, Hongru He, Xuanli Wong, Kam-Fai Minervini, Pasquale |
| contents | Large language models (LLMs) can store a significant amount of factual knowledge in their parameters. However, their parametric knowledge may conflict with the information provided in the context -- this phenomenon, known as \emph{context-memory knowledge conflicts}, can lead to undesirable model behaviour, such as reliance on outdated or incorrect information. Analysing the internal activations of LLMs, we find that they can internally register the signals of knowledge conflict at mid-layers. Such signals allow us to detect whether a knowledge conflict occurs and use \emph{inference-time} intervention strategies to resolve it. In this work, we propose \textsc{SpARE}, a \emph{training-free} representation engineering method that uses pre-trained sparse auto-encoders (SAEs) to control the knowledge selection behaviour of LLMs. \textsc{SpARE} identifies the functional features that control the knowledge selection behaviours and applies them to edit the internal activations of LLMs at inference time. Our experimental results show that \textsc{SpARE} can effectively control the usage of either knowledge source to resolve knowledge conflict in open-domain question-answering tasks, surpassing existing representation engineering methods ($+10\%$) as well as contrastive decoding methods ($+15\%$). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_15999 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering Zhao, Yu Devoto, Alessio Hong, Giwon Du, Xiaotang Gema, Aryo Pradipta Wang, Hongru He, Xuanli Wong, Kam-Fai Minervini, Pasquale Computation and Language Large language models (LLMs) can store a significant amount of factual knowledge in their parameters. However, their parametric knowledge may conflict with the information provided in the context -- this phenomenon, known as \emph{context-memory knowledge conflicts}, can lead to undesirable model behaviour, such as reliance on outdated or incorrect information. Analysing the internal activations of LLMs, we find that they can internally register the signals of knowledge conflict at mid-layers. Such signals allow us to detect whether a knowledge conflict occurs and use \emph{inference-time} intervention strategies to resolve it. In this work, we propose \textsc{SpARE}, a \emph{training-free} representation engineering method that uses pre-trained sparse auto-encoders (SAEs) to control the knowledge selection behaviour of LLMs. \textsc{SpARE} identifies the functional features that control the knowledge selection behaviours and applies them to edit the internal activations of LLMs at inference time. Our experimental results show that \textsc{SpARE} can effectively control the usage of either knowledge source to resolve knowledge conflict in open-domain question-answering tasks, surpassing existing representation engineering methods ($+10\%$) as well as contrastive decoding methods ($+15\%$). |
| title | Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.15999 |