Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.09630 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910005135933440 |
|---|---|
| author | Gaintseva, Tatiana Oncescu, Andreea-Maria Ma, Chengcheng Liu, Ziquan Benning, Martin Slabaugh, Gregory Deng, Jiankang Elezi, Ismail |
| author_facet | Gaintseva, Tatiana Oncescu, Andreea-Maria Ma, Chengcheng Liu, Ziquan Benning, Martin Slabaugh, Gregory Deng, Jiankang Elezi, Ismail |
| contents | Diffusion models have transformed image generation, yet controlling their outputs to reliably erase undesired concepts remains challenging. Existing approaches usually require task-specific training and struggle to generalize across both concrete (e.g., objects) and abstract (e.g., styles) concepts. We propose CASteer (Cross-Attention Steering), a training-free framework for concept erasure in diffusion models using steering vectors to influence hidden representations dynamically. CASteer precomputes concept-specific steering vectors by averaging neural activations from images generated for each target concept. During inference, it dynamically applies these vectors to suppress undesired concepts only when they appear, ensuring that unrelated regions remain unaffected. This selective activation enables precise, context-aware erasure without degrading overall image quality. This approach achieves effective removal of harmful or unwanted content across a wide range of visual concepts, all without model retraining. CASteer outperforms state-of-the-art concept erasure techniques while preserving unrelated content and minimizing unintended effects. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_09630 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CASteer: Cross-Attention Steering for Controllable Concept Erasure Gaintseva, Tatiana Oncescu, Andreea-Maria Ma, Chengcheng Liu, Ziquan Benning, Martin Slabaugh, Gregory Deng, Jiankang Elezi, Ismail Graphics Diffusion models have transformed image generation, yet controlling their outputs to reliably erase undesired concepts remains challenging. Existing approaches usually require task-specific training and struggle to generalize across both concrete (e.g., objects) and abstract (e.g., styles) concepts. We propose CASteer (Cross-Attention Steering), a training-free framework for concept erasure in diffusion models using steering vectors to influence hidden representations dynamically. CASteer precomputes concept-specific steering vectors by averaging neural activations from images generated for each target concept. During inference, it dynamically applies these vectors to suppress undesired concepts only when they appear, ensuring that unrelated regions remain unaffected. This selective activation enables precise, context-aware erasure without degrading overall image quality. This approach achieves effective removal of harmful or unwanted content across a wide range of visual concepts, all without model retraining. CASteer outperforms state-of-the-art concept erasure techniques while preserving unrelated content and minimizing unintended effects. |
| title | CASteer: Cross-Attention Steering for Controllable Concept Erasure |
| topic | Graphics |
| url | https://arxiv.org/abs/2503.09630 |