Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.28553 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917540478844928 |
|---|---|
| author | Collu, Matteo Gioele Conte, Riccardo Giaretta, Alberto Kleyko, Denis Conti, Mauro Zavatteri, Matteo Confalonieri, Roberto |
| author_facet | Collu, Matteo Gioele Conte, Riccardo Giaretta, Alberto Kleyko, Denis Conti, Mauro Zavatteri, Matteo Confalonieri, Roberto |
| contents | In this paper, we investigate whether refusal behavior can be predicted from LLM intermediate activations before decoding using linear probes trained on residual stream activations at each transformer block. We find that refusal is linearly decodable well before the final layer, indicating that safety-relevant behavior is represented in intermediate activations before output generation. To test whether this signal is actionable, we introduce Mechanistic AutoDAN, a probe-guided variant of AutoDAN that replaces full-model fitness evaluation with partial forward passes and probe-based scoring inside a genetic prompt search loop. Across the evaluated models, our method achieves attack success rates competitive with vanilla AutoDAN while reducing per-iteration search time by up to 72%, and probe-guided prompts match or exceed AutoDAN's cross-model transfer in several configurations. We further find that the usefulness of probe guidance increases with model scale. Our results show that refusal is not only observable at the output level, but is encoded as a structured and actionable signal in intermediate LLM activations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_28553 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activations Collu, Matteo Gioele Conte, Riccardo Giaretta, Alberto Kleyko, Denis Conti, Mauro Zavatteri, Matteo Confalonieri, Roberto Artificial Intelligence Cryptography and Security In this paper, we investigate whether refusal behavior can be predicted from LLM intermediate activations before decoding using linear probes trained on residual stream activations at each transformer block. We find that refusal is linearly decodable well before the final layer, indicating that safety-relevant behavior is represented in intermediate activations before output generation. To test whether this signal is actionable, we introduce Mechanistic AutoDAN, a probe-guided variant of AutoDAN that replaces full-model fitness evaluation with partial forward passes and probe-based scoring inside a genetic prompt search loop. Across the evaluated models, our method achieves attack success rates competitive with vanilla AutoDAN while reducing per-iteration search time by up to 72%, and probe-guided prompts match or exceed AutoDAN's cross-model transfer in several configurations. We further find that the usefulness of probe guidance increases with model scale. Our results show that refusal is not only observable at the output level, but is encoded as a structured and actionable signal in intermediate LLM activations. |
| title | Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activations |
| topic | Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2605.28553 |