Task and Perception-aware Distributed Source Coding for Correlated Speech under Bandwidth-constrained Channels
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910805568520192 |
|---|---|
| author | Bhattacharya, Sagnik Mohsin, Muhammad Ahmed Bilal, Ahsan Cioffi, John M. |
| author_facet | Bhattacharya, Sagnik Mohsin, Muhammad Ahmed Bilal, Ahsan Cioffi, John M. |
| contents | Emerging wireless AR/VR applications require real-time transmission of correlated high-fidelity speech from multiple resource-constrained devices over unreliable, bandwidth-limited channels. Existing autoencoder-based speech source coding methods fail to address the combination of the following - (1) dynamic bitrate adaptation without retraining the model, (2) leveraging correlations among multiple speech sources, and (3) balancing downstream task loss with realism of reconstructed speech. We propose a neural distributed principal component analysis (NDPCA)-aided distributed source coding algorithm for correlated speech sources transmitting to a central receiver. Our method includes a perception-aware downstream task loss function that balances perceptual realism with task-specific performance. Experiments show significant PSNR improvements under bandwidth constraints over naive autoencoder methods in task-agnostic (19%) and task-aware settings (52%). It also approaches the theoretical upper bound, where all correlated sources are sent to a single encoder, especially in low-bandwidth scenarios. Additionally, we present a rate-distortion-perception trade-off curve, enabling adaptive decisions based on application-specific realism needs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_17879 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Task and Perception-aware Distributed Source Coding for Correlated Speech under Bandwidth-constrained Channels Bhattacharya, Sagnik Mohsin, Muhammad Ahmed Bilal, Ahsan Cioffi, John M. Information Theory Artificial Intelligence Sound Audio and Speech Processing Signal Processing Emerging wireless AR/VR applications require real-time transmission of correlated high-fidelity speech from multiple resource-constrained devices over unreliable, bandwidth-limited channels. Existing autoencoder-based speech source coding methods fail to address the combination of the following - (1) dynamic bitrate adaptation without retraining the model, (2) leveraging correlations among multiple speech sources, and (3) balancing downstream task loss with realism of reconstructed speech. We propose a neural distributed principal component analysis (NDPCA)-aided distributed source coding algorithm for correlated speech sources transmitting to a central receiver. Our method includes a perception-aware downstream task loss function that balances perceptual realism with task-specific performance. Experiments show significant PSNR improvements under bandwidth constraints over naive autoencoder methods in task-agnostic (19%) and task-aware settings (52%). It also approaches the theoretical upper bound, where all correlated sources are sent to a single encoder, especially in low-bandwidth scenarios. Additionally, we present a rate-distortion-perception trade-off curve, enabling adaptive decisions based on application-specific realism needs. |
| title | Task and Perception-aware Distributed Source Coding for Correlated Speech under Bandwidth-constrained Channels |
| topic | Information Theory Artificial Intelligence Sound Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2501.17879 |