EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917882875609088 |
|---|---|
| author | Gao, Daiheng Lu, Shilin Walters, Shaw Zhou, Wenbo Chu, Jiaming Zhang, Jie Zhang, Bang Jia, Mengxi Zhao, Jian Fan, Zhaoxin Zhang, Weiming |
| author_facet | Gao, Daiheng Lu, Shilin Walters, Shaw Zhou, Wenbo Chu, Jiaming Zhang, Jie Zhang, Bang Jia, Mengxi Zhao, Jian Fan, Zhaoxin Zhang, Weiming |
| contents | Removing unwanted concepts from large-scale text-to-image (T2I) diffusion models while maintaining their overall generative quality remains an open challenge. This difficulty is especially pronounced in emerging paradigms, such as Stable Diffusion (SD) v3 and Flux, which incorporate flow matching and transformer-based architectures. These advancements limit the transferability of existing concept-erasure techniques that were originally designed for the previous T2I paradigm (e.g., SD v1.4). In this work, we introduce EraseAnything, the first method specifically developed to address concept erasure within the latest flow-based T2I framework. We formulate concept erasure as a bi-level optimization problem, employing LoRA-based parameter tuning and an attention map regularizer to selectively suppress undesirable activations. Furthermore, we propose a self-contrastive learning strategy to ensure that removing unwanted concepts does not inadvertently harm performance on unrelated ones. Experimental results demonstrate that EraseAnything successfully fills the research gap left by earlier methods in this new T2I paradigm, achieving state-of-the-art performance across a wide range of concept erasure tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_20413 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers Gao, Daiheng Lu, Shilin Walters, Shaw Zhou, Wenbo Chu, Jiaming Zhang, Jie Zhang, Bang Jia, Mengxi Zhao, Jian Fan, Zhaoxin Zhang, Weiming Computer Vision and Pattern Recognition Removing unwanted concepts from large-scale text-to-image (T2I) diffusion models while maintaining their overall generative quality remains an open challenge. This difficulty is especially pronounced in emerging paradigms, such as Stable Diffusion (SD) v3 and Flux, which incorporate flow matching and transformer-based architectures. These advancements limit the transferability of existing concept-erasure techniques that were originally designed for the previous T2I paradigm (e.g., SD v1.4). In this work, we introduce EraseAnything, the first method specifically developed to address concept erasure within the latest flow-based T2I framework. We formulate concept erasure as a bi-level optimization problem, employing LoRA-based parameter tuning and an attention map regularizer to selectively suppress undesirable activations. Furthermore, we propose a self-contrastive learning strategy to ensure that removing unwanted concepts does not inadvertently harm performance on unrelated ones. Experimental results demonstrate that EraseAnything successfully fills the research gap left by earlier methods in this new T2I paradigm, achieving state-of-the-art performance across a wide range of concept erasure tasks. |
| title | EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.20413 |