Naturalistic Music Decoding from EEG Data via Latent Diffusion Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913641901588480 |
|---|---|
| author | Postolache, Emilian Polouliakh, Natalia Kitano, Hiroaki Connelly, Akima Rodolà, Emanuele Cosmo, Luca Akama, Taketo |
| author_facet | Postolache, Emilian Polouliakh, Natalia Kitano, Hiroaki Connelly, Akima Rodolà, Emanuele Cosmo, Luca Akama, Taketo |
| contents | In this article, we explore the potential of using latent diffusion models, a family of powerful generative models, for the task of reconstructing naturalistic music from electroencephalogram (EEG) recordings. Unlike simpler music with limited timbres, such as MIDI-generated tunes or monophonic pieces, the focus here is on intricate music featuring a diverse array of instruments, voices, and effects, rich in harmonics and timbre. This study represents an initial foray into achieving general music reconstruction of high-quality using non-invasive EEG data, employing an end-to-end training approach directly on raw data without the need for manual pre-processing and channel selection. We train our models on the public NMED-T dataset and perform quantitative evaluation proposing neural embedding-based metrics. Our work contributes to the ongoing research in neural decoding and brain-computer interfaces, offering insights into the feasibility of using EEG data for complex auditory information reconstruction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_09062 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Naturalistic Music Decoding from EEG Data via Latent Diffusion Models Postolache, Emilian Polouliakh, Natalia Kitano, Hiroaki Connelly, Akima Rodolà, Emanuele Cosmo, Luca Akama, Taketo Sound Machine Learning Audio and Speech Processing In this article, we explore the potential of using latent diffusion models, a family of powerful generative models, for the task of reconstructing naturalistic music from electroencephalogram (EEG) recordings. Unlike simpler music with limited timbres, such as MIDI-generated tunes or monophonic pieces, the focus here is on intricate music featuring a diverse array of instruments, voices, and effects, rich in harmonics and timbre. This study represents an initial foray into achieving general music reconstruction of high-quality using non-invasive EEG data, employing an end-to-end training approach directly on raw data without the need for manual pre-processing and channel selection. We train our models on the public NMED-T dataset and perform quantitative evaluation proposing neural embedding-based metrics. Our work contributes to the ongoing research in neural decoding and brain-computer interfaces, offering insights into the feasibility of using EEG data for complex auditory information reconstruction. |
| title | Naturalistic Music Decoding from EEG Data via Latent Diffusion Models |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2405.09062 |