Secure & Personalized Music-to-Video Generation via CHARCHA
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915138127265792 |
|---|---|
| author | Agarwal, Mehul Agarwal, Gauri Benoit, Santiago Lippman, Andrew Oh, Jean |
| author_facet | Agarwal, Mehul Agarwal, Gauri Benoit, Santiago Lippman, Andrew Oh, Jean |
| contents | Music is a deeply personal experience and our aim is to enhance this with a fully-automated pipeline for personalized music video generation. Our work allows listeners to not just be consumers but co-creators in the music video generation process by creating personalized, consistent and context-driven visuals based on lyrics, rhythm and emotion in the music. The pipeline combines multimodal translation and generation techniques and utilizes low-rank adaptation on listeners' images to create immersive music videos that reflect both the music and the individual. To ensure the ethical use of users' identity, we also introduce CHARCHA (patent pending), a facial identity verification protocol that protects people against unauthorized use of their face while at the same time collecting authorized images from users for personalizing their videos. This paper thus provides a secure and innovative framework for creating deeply personalized music videos. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_02610 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Secure & Personalized Music-to-Video Generation via CHARCHA Agarwal, Mehul Agarwal, Gauri Benoit, Santiago Lippman, Andrew Oh, Jean Artificial Intelligence Computer Vision and Pattern Recognition Human-Computer Interaction Multimedia Music is a deeply personal experience and our aim is to enhance this with a fully-automated pipeline for personalized music video generation. Our work allows listeners to not just be consumers but co-creators in the music video generation process by creating personalized, consistent and context-driven visuals based on lyrics, rhythm and emotion in the music. The pipeline combines multimodal translation and generation techniques and utilizes low-rank adaptation on listeners' images to create immersive music videos that reflect both the music and the individual. To ensure the ethical use of users' identity, we also introduce CHARCHA (patent pending), a facial identity verification protocol that protects people against unauthorized use of their face while at the same time collecting authorized images from users for personalizing their videos. This paper thus provides a secure and innovative framework for creating deeply personalized music videos. |
| title | Secure & Personalized Music-to-Video Generation via CHARCHA |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition Human-Computer Interaction Multimedia |
| url | https://arxiv.org/abs/2502.02610 |