Understanding Refusal in Language Models with Sparse Autoencoders
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909627287863296 |
|---|---|
| author | Yeo, Wei Jie Prakash, Nirmalendu Neo, Clement Lee, Roy Ka-Wei Cambria, Erik Satapathy, Ranjan |
| author_facet | Yeo, Wei Jie Prakash, Nirmalendu Neo, Clement Lee, Roy Ka-Wei Cambria, Erik Satapathy, Ranjan |
| contents | Refusal is a key safety behavior in aligned language models, yet the internal mechanisms driving refusals remain opaque. In this work, we conduct a mechanistic study of refusal in instruction-tuned LLMs using sparse autoencoders to identify latent features that causally mediate refusal behaviors. We apply our method to two open-source chat models and intervene on refusal-related features to assess their influence on generation, validating their behavioral impact across multiple harmful datasets. This enables a fine-grained inspection of how refusal manifests at the activation level and addresses key research questions such as investigating upstream-downstream latent relationship and understanding the mechanisms of adversarial jailbreaking techniques. We also establish the usefulness of refusal features in enhancing generalization for linear probes to out-of-distribution adversarial samples in classification tasks. We open source our code in https://github.com/wj210/refusal_sae. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23556 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Understanding Refusal in Language Models with Sparse Autoencoders Yeo, Wei Jie Prakash, Nirmalendu Neo, Clement Lee, Roy Ka-Wei Cambria, Erik Satapathy, Ranjan Computation and Language Refusal is a key safety behavior in aligned language models, yet the internal mechanisms driving refusals remain opaque. In this work, we conduct a mechanistic study of refusal in instruction-tuned LLMs using sparse autoencoders to identify latent features that causally mediate refusal behaviors. We apply our method to two open-source chat models and intervene on refusal-related features to assess their influence on generation, validating their behavioral impact across multiple harmful datasets. This enables a fine-grained inspection of how refusal manifests at the activation level and addresses key research questions such as investigating upstream-downstream latent relationship and understanding the mechanisms of adversarial jailbreaking techniques. We also establish the usefulness of refusal features in enhancing generalization for linear probes to out-of-distribution adversarial samples in classification tasks. We open source our code in https://github.com/wj210/refusal_sae. |
| title | Understanding Refusal in Language Models with Sparse Autoencoders |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.23556 |