DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909406034132992 |
|---|---|
| author | Aiello, Emanuele Michieli, Umberto Valsesia, Diego Ozay, Mete Magli, Enrico |
| author_facet | Aiello, Emanuele Michieli, Umberto Valsesia, Diego Ozay, Mete Magli, Enrico |
| contents | Personalized image generation requires text-to-image generative models that capture the core features of a reference subject to allow for controlled generation across different contexts. Existing methods face challenges due to complex training requirements, high inference costs, limited flexibility, or a combination of these issues. In this paper, we introduce DreamCache, a scalable approach for efficient and high-quality personalized image generation. By caching a small number of reference image features from a subset of layers and a single timestep of the pretrained diffusion denoiser, DreamCache enables dynamic modulation of the generated image features through lightweight, trained conditioning adapters. DreamCache achieves state-of-the-art image and text alignment, utilizing an order of magnitude fewer extra parameters, and is both more computationally effective and versatile than existing models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_17786 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching Aiello, Emanuele Michieli, Umberto Valsesia, Diego Ozay, Mete Magli, Enrico Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Personalized image generation requires text-to-image generative models that capture the core features of a reference subject to allow for controlled generation across different contexts. Existing methods face challenges due to complex training requirements, high inference costs, limited flexibility, or a combination of these issues. In this paper, we introduce DreamCache, a scalable approach for efficient and high-quality personalized image generation. By caching a small number of reference image features from a subset of layers and a single timestep of the pretrained diffusion denoiser, DreamCache enables dynamic modulation of the generated image features through lightweight, trained conditioning adapters. DreamCache achieves state-of-the-art image and text alignment, utilizing an order of magnitude fewer extra parameters, and is both more computationally effective and versatile than existing models. |
| title | DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2411.17786 |