ReFACT: Updating Text-to-Image Models by Editing the Text Encoder
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911868933636096 |
|---|---|
| author | Arad, Dana Orgad, Hadas Belinkov, Yonatan |
| author_facet | Arad, Dana Orgad, Hadas Belinkov, Yonatan |
| contents | Our world is marked by unprecedented technological, global, and socio-political transformations, posing a significant challenge to text-to-image generative models. These models encode factual associations within their parameters that can quickly become outdated, diminishing their utility for end-users. To that end, we introduce ReFACT, a novel approach for editing factual associations in text-to-image models without relaying on explicit input from end-users or costly re-training. ReFACT updates the weights of a specific layer in the text encoder, modifying only a tiny portion of the model's parameters and leaving the rest of the model unaffected. We empirically evaluate ReFACT on an existing benchmark, alongside a newly curated dataset. Compared to other methods, ReFACT achieves superior performance in both generalization to related concepts and preservation of unrelated concepts. Furthermore, ReFACT maintains image generation quality, making it a practical tool for updating and correcting factual information in text-to-image models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_00738 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | ReFACT: Updating Text-to-Image Models by Editing the Text Encoder Arad, Dana Orgad, Hadas Belinkov, Yonatan Computation and Language Computer Vision and Pattern Recognition 68T50 I.2.7 Our world is marked by unprecedented technological, global, and socio-political transformations, posing a significant challenge to text-to-image generative models. These models encode factual associations within their parameters that can quickly become outdated, diminishing their utility for end-users. To that end, we introduce ReFACT, a novel approach for editing factual associations in text-to-image models without relaying on explicit input from end-users or costly re-training. ReFACT updates the weights of a specific layer in the text encoder, modifying only a tiny portion of the model's parameters and leaving the rest of the model unaffected. We empirically evaluate ReFACT on an existing benchmark, alongside a newly curated dataset. Compared to other methods, ReFACT achieves superior performance in both generalization to related concepts and preservation of unrelated concepts. Furthermore, ReFACT maintains image generation quality, making it a practical tool for updating and correcting factual information in text-to-image models. |
| title | ReFACT: Updating Text-to-Image Models by Editing the Text Encoder |
| topic | Computation and Language Computer Vision and Pattern Recognition 68T50 I.2.7 |
| url | https://arxiv.org/abs/2306.00738 |