Diff4Steer: Steerable Diffusion Prior for Generative Music Retrieval with Semantic Guidance
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909590735552512 |
|---|---|
| author | Bao, Xuchan Li, Judith Yue Wan, Zhong Yi Su, Kun Denk, Timo Lee, Joonseok Kuzmin, Dima Sha, Fei |
| author_facet | Bao, Xuchan Li, Judith Yue Wan, Zhong Yi Su, Kun Denk, Timo Lee, Joonseok Kuzmin, Dima Sha, Fei |
| contents | Modern music retrieval systems often rely on fixed representations of user preferences, limiting their ability to capture users' diverse and uncertain retrieval needs. To address this limitation, we introduce Diff4Steer, a novel generative retrieval framework that employs lightweight diffusion models to synthesize diverse seed embeddings from user queries that represent potential directions for music exploration. Unlike deterministic methods that map user query to a single point in embedding space, Diff4Steer provides a statistical prior on the target modality (audio) for retrieval, effectively capturing the uncertainty and multi-faceted nature of user preferences. Furthermore, Diff4Steer can be steered by image or text inputs, enabling more flexible and controllable music discovery combined with nearest neighbor search. Our framework outperforms deterministic regression methods and LLM-based generative retrieval baseline in terms of retrieval and ranking metrics, demonstrating its effectiveness in capturing user preferences, leading to more diverse and relevant recommendations. Listening examples are available at tinyurl.com/diff4steer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04746 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Diff4Steer: Steerable Diffusion Prior for Generative Music Retrieval with Semantic Guidance Bao, Xuchan Li, Judith Yue Wan, Zhong Yi Su, Kun Denk, Timo Lee, Joonseok Kuzmin, Dima Sha, Fei Sound Information Retrieval Multimedia Audio and Speech Processing Modern music retrieval systems often rely on fixed representations of user preferences, limiting their ability to capture users' diverse and uncertain retrieval needs. To address this limitation, we introduce Diff4Steer, a novel generative retrieval framework that employs lightweight diffusion models to synthesize diverse seed embeddings from user queries that represent potential directions for music exploration. Unlike deterministic methods that map user query to a single point in embedding space, Diff4Steer provides a statistical prior on the target modality (audio) for retrieval, effectively capturing the uncertainty and multi-faceted nature of user preferences. Furthermore, Diff4Steer can be steered by image or text inputs, enabling more flexible and controllable music discovery combined with nearest neighbor search. Our framework outperforms deterministic regression methods and LLM-based generative retrieval baseline in terms of retrieval and ranking metrics, demonstrating its effectiveness in capturing user preferences, leading to more diverse and relevant recommendations. Listening examples are available at tinyurl.com/diff4steer. |
| title | Diff4Steer: Steerable Diffusion Prior for Generative Music Retrieval with Semantic Guidance |
| topic | Sound Information Retrieval Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2412.04746 |