Compression of Higher Order Ambisonics with Multichannel RVQGAN
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912152923668480 |
|---|---|
| author | Hirvonen, Toni Namazi, Mahmoud |
| author_facet | Hirvonen, Toni Namazi, Mahmoud |
| contents | A multichannel extension to the RVQGAN neural coding method is proposed, and realized for data-driven compression of third-order Ambisonics audio. The input- and output layers of the generator and discriminator models are modified to accept multiple (16) channels without increasing the model bitrate. We also propose a loss function for accounting for spatial perception in immersive reproduction, and transfer learning from single-channel models. Listening test results with 7.1.4 immersive playback show that the proposed extension is suitable for coding scene-based, 16-channel Ambisonics content with good quality at 16 kbps when trained and tested on the EigenScape database. The model has potential applications for learning other types of content and multichannel formats. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_12008 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Compression of Higher Order Ambisonics with Multichannel RVQGAN Hirvonen, Toni Namazi, Mahmoud Sound Machine Learning Multimedia Audio and Speech Processing A multichannel extension to the RVQGAN neural coding method is proposed, and realized for data-driven compression of third-order Ambisonics audio. The input- and output layers of the generator and discriminator models are modified to accept multiple (16) channels without increasing the model bitrate. We also propose a loss function for accounting for spatial perception in immersive reproduction, and transfer learning from single-channel models. Listening test results with 7.1.4 immersive playback show that the proposed extension is suitable for coding scene-based, 16-channel Ambisonics content with good quality at 16 kbps when trained and tested on the EigenScape database. The model has potential applications for learning other types of content and multichannel formats. |
| title | Compression of Higher Order Ambisonics with Multichannel RVQGAN |
| topic | Sound Machine Learning Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2411.12008 |