The Fabrication of Reality and Fantasy: Scene Generation with LLM-Assisted Prompt Interpretation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914875066810368 |
|---|---|
| author | Yao, Yi Hsu, Chan-Feng Lin, Jhe-Hao Xie, Hongxia Lin, Terence Huang, Yi-Ning Shuai, Hong-Han Cheng, Wen-Huang |
| author_facet | Yao, Yi Hsu, Chan-Feng Lin, Jhe-Hao Xie, Hongxia Lin, Terence Huang, Yi-Ning Shuai, Hong-Han Cheng, Wen-Huang |
| contents | In spite of recent advancements in text-to-image generation, limitations persist in handling complex and imaginative prompts due to the restricted diversity and complexity of training data. This work explores how diffusion models can generate images from prompts requiring artistic creativity or specialized knowledge. We introduce the Realistic-Fantasy Benchmark (RFBench), a novel evaluation framework blending realistic and fantastical scenarios. To address these challenges, we propose the Realistic-Fantasy Network (RFNet), a training-free approach integrating diffusion models with LLMs. Extensive human evaluations and GPT-based compositional assessments demonstrate our approach's superiority over state-of-the-art methods. Our code and dataset is available at https://leo81005.github.io/Reality-and-Fantasy/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12579 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Fabrication of Reality and Fantasy: Scene Generation with LLM-Assisted Prompt Interpretation Yao, Yi Hsu, Chan-Feng Lin, Jhe-Hao Xie, Hongxia Lin, Terence Huang, Yi-Ning Shuai, Hong-Han Cheng, Wen-Huang Computer Vision and Pattern Recognition Artificial Intelligence In spite of recent advancements in text-to-image generation, limitations persist in handling complex and imaginative prompts due to the restricted diversity and complexity of training data. This work explores how diffusion models can generate images from prompts requiring artistic creativity or specialized knowledge. We introduce the Realistic-Fantasy Benchmark (RFBench), a novel evaluation framework blending realistic and fantastical scenarios. To address these challenges, we propose the Realistic-Fantasy Network (RFNet), a training-free approach integrating diffusion models with LLMs. Extensive human evaluations and GPT-based compositional assessments demonstrate our approach's superiority over state-of-the-art methods. Our code and dataset is available at https://leo81005.github.io/Reality-and-Fantasy/. |
| title | The Fabrication of Reality and Fantasy: Scene Generation with LLM-Assisted Prompt Interpretation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2407.12579 |