RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909472609271808 |
|---|---|
| author | Cao, Yuefan Gong, Chengyue Li, Xiaoyu Liang, Yingyu Sha, Zhizhou Shi, Zhenmei Song, Zhao |
| author_facet | Cao, Yuefan Gong, Chengyue Li, Xiaoyu Liang, Yingyu Sha, Zhizhou Shi, Zhenmei Song, Zhao |
| contents | Text-to-video generation models have made impressive progress, but they still struggle with generating videos with complex features. This limitation often arises from the inability of the text encoder to produce accurate embeddings, which hinders the video generation model. In this work, we propose a novel approach to overcome this challenge by selecting the optimal text embedding through interpolation in the embedding space. We demonstrate that this method enables the video generation model to produce the desired videos. Additionally, we introduce a simple algorithm using perpendicular foot embeddings and cosine similarity to identify the optimal interpolation embedding. Our findings highlight the importance of accurate text embeddings and offer a pathway for improving text-to-video generation performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_09982 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation Cao, Yuefan Gong, Chengyue Li, Xiaoyu Liang, Yingyu Sha, Zhizhou Shi, Zhenmei Song, Zhao Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning Text-to-video generation models have made impressive progress, but they still struggle with generating videos with complex features. This limitation often arises from the inability of the text encoder to produce accurate embeddings, which hinders the video generation model. In this work, we propose a novel approach to overcome this challenge by selecting the optimal text embedding through interpolation in the embedding space. We demonstrate that this method enables the video generation model to produce the desired videos. Additionally, we introduce a simple algorithm using perpendicular foot embeddings and cosine similarity to identify the optimal interpolation embedding. Our findings highlight the importance of accurate text embeddings and offer a pathway for improving text-to-video generation performance. |
| title | RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2501.09982 |