LAV: Audio-Driven Dynamic Visual Generation with Neural Compression and StyleGAN2
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918446596358144 |
|---|---|
| author | Jung, Jongmin Jeong, Dasaem |
| author_facet | Jung, Jongmin Jeong, Dasaem |
| contents | This paper introduces LAV (Latent Audio-Visual), a system that integrates EnCodec's neural audio compression with StyleGAN2's generative capabilities to produce visually dynamic outputs driven by pre-recorded audio. Unlike previous works that rely on explicit feature mappings, LAV uses EnCodec embeddings as latent representations, directly transformed into StyleGAN2's style latent space via randomly initialized linear mapping. This approach preserves semantic richness in the transformation, enabling nuanced and semantically coherent audio-visual translations. The framework demonstrates the potential of using pretrained audio compression models for artistic and computational applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_10101 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LAV: Audio-Driven Dynamic Visual Generation with Neural Compression and StyleGAN2 Jung, Jongmin Jeong, Dasaem Sound Artificial Intelligence Graphics Multimedia Audio and Speech Processing This paper introduces LAV (Latent Audio-Visual), a system that integrates EnCodec's neural audio compression with StyleGAN2's generative capabilities to produce visually dynamic outputs driven by pre-recorded audio. Unlike previous works that rely on explicit feature mappings, LAV uses EnCodec embeddings as latent representations, directly transformed into StyleGAN2's style latent space via randomly initialized linear mapping. This approach preserves semantic richness in the transformation, enabling nuanced and semantically coherent audio-visual translations. The framework demonstrates the potential of using pretrained audio compression models for artistic and computational applications. |
| title | LAV: Audio-Driven Dynamic Visual Generation with Neural Compression and StyleGAN2 |
| topic | Sound Artificial Intelligence Graphics Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.10101 |