Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transport
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913195888738304 |
|---|---|
| author | Torres, Bernardo Peeters, Geoffroy Richard, Gaël |
| author_facet | Torres, Bernardo Peeters, Geoffroy Richard, Gaël |
| contents | In neural audio signal processing, pitch conditioning has been used to enhance the performance of synthesizers. However, jointly training pitch estimators and synthesizers is a challenge when using standard audio-to-audio reconstruction loss, leading to reliance on external pitch trackers. To address this issue, we propose using a spectral loss function inspired by optimal transportation theory that minimizes the displacement of spectral energy. We validate this approach through an unsupervised autoencoding task that fits a harmonic template to harmonic signals. We jointly estimate the fundamental frequency and amplitudes of harmonics using a lightweight encoder and reconstruct the signals using a differentiable harmonic synthesizer. The proposed approach offers a promising direction for improving unsupervised parameter estimation in neural audio applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_14507 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transport Torres, Bernardo Peeters, Geoffroy Richard, Gaël Sound Machine Learning Audio and Speech Processing Signal Processing In neural audio signal processing, pitch conditioning has been used to enhance the performance of synthesizers. However, jointly training pitch estimators and synthesizers is a challenge when using standard audio-to-audio reconstruction loss, leading to reliance on external pitch trackers. To address this issue, we propose using a spectral loss function inspired by optimal transportation theory that minimizes the displacement of spectral energy. We validate this approach through an unsupervised autoencoding task that fits a harmonic template to harmonic signals. We jointly estimate the fundamental frequency and amplitudes of harmonics using a lightweight encoder and reconstruct the signals using a differentiable harmonic synthesizer. The proposed approach offers a promising direction for improving unsupervised parameter estimation in neural audio applications. |
| title | Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transport |
| topic | Sound Machine Learning Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2312.14507 |