InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909163008819200 |
|---|---|
| author | Guo, Xiefan Liu, Jinlin Cui, Miaomiao Li, Jiankai Yang, Hongyu Huang, Di |
| author_facet | Guo, Xiefan Liu, Jinlin Cui, Miaomiao Li, Jiankai Yang, Hongyu Huang, Di |
| contents | Recent strides in the development of diffusion models, exemplified by advancements such as Stable Diffusion, have underscored their remarkable prowess in generating visually compelling images. However, the imperative of achieving a seamless alignment between the generated image and the provided prompt persists as a formidable challenge. This paper traces the root of these difficulties to invalid initial noise, and proposes a solution in the form of Initial Noise Optimization (InitNO), a paradigm that refines this noise. Considering text prompts, not all random noises are effective in synthesizing semantically-faithful images. We design the cross-attention response score and the self-attention conflict score to evaluate the initial noise, bifurcating the initial latent space into valid and invalid sectors. A strategically crafted noise optimization pipeline is developed to guide the initial noise towards valid regions. Our method, validated through rigorous experimentation, shows a commendable proficiency in generating images in strict accordance with text prompts. Our code is available at https://github.com/xiefan-guo/initno. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_04650 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization Guo, Xiefan Liu, Jinlin Cui, Miaomiao Li, Jiankai Yang, Hongyu Huang, Di Computer Vision and Pattern Recognition Recent strides in the development of diffusion models, exemplified by advancements such as Stable Diffusion, have underscored their remarkable prowess in generating visually compelling images. However, the imperative of achieving a seamless alignment between the generated image and the provided prompt persists as a formidable challenge. This paper traces the root of these difficulties to invalid initial noise, and proposes a solution in the form of Initial Noise Optimization (InitNO), a paradigm that refines this noise. Considering text prompts, not all random noises are effective in synthesizing semantically-faithful images. We design the cross-attention response score and the self-attention conflict score to evaluate the initial noise, bifurcating the initial latent space into valid and invalid sectors. A strategically crafted noise optimization pipeline is developed to guide the initial noise towards valid regions. Our method, validated through rigorous experimentation, shows a commendable proficiency in generating images in strict accordance with text prompts. Our code is available at https://github.com/xiefan-guo/initno. |
| title | InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.04650 |