HybridBooth: Hybrid Prompt Inversion for Efficient Subject-Driven Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910644266074112 |
|---|---|
| author | Guan, Shanyan Ge, Yanhao Tai, Ying Yang, Jian Li, Wei You, Mingyu |
| author_facet | Guan, Shanyan Ge, Yanhao Tai, Ying Yang, Jian Li, Wei You, Mingyu |
| contents | Recent advancements in text-to-image diffusion models have shown remarkable creative capabilities with textual prompts, but generating personalized instances based on specific subjects, known as subject-driven generation, remains challenging. To tackle this issue, we present a new hybrid framework called HybridBooth, which merges the benefits of optimization-based and direct-regression methods. HybridBooth operates in two stages: the Word Embedding Probe, which generates a robust initial word embedding using a fine-tuned encoder, and the Word Embedding Refinement, which further adapts the encoder to specific subject images by optimizing key parameters. This approach allows for effective and fast inversion of visual concepts into textual embedding, even from a single image, while maintaining the model's generalization capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08192 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HybridBooth: Hybrid Prompt Inversion for Efficient Subject-Driven Generation Guan, Shanyan Ge, Yanhao Tai, Ying Yang, Jian Li, Wei You, Mingyu Computer Vision and Pattern Recognition Recent advancements in text-to-image diffusion models have shown remarkable creative capabilities with textual prompts, but generating personalized instances based on specific subjects, known as subject-driven generation, remains challenging. To tackle this issue, we present a new hybrid framework called HybridBooth, which merges the benefits of optimization-based and direct-regression methods. HybridBooth operates in two stages: the Word Embedding Probe, which generates a robust initial word embedding using a fine-tuned encoder, and the Word Embedding Refinement, which further adapts the encoder to specific subject images by optimizing key parameters. This approach allows for effective and fast inversion of visual concepts into textual embedding, even from a single image, while maintaining the model's generalization capabilities. |
| title | HybridBooth: Hybrid Prompt Inversion for Efficient Subject-Driven Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.08192 |