Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914636718145536 |
|---|---|
| author | Inoue, Koji Jiang, Bing'er Ekstedt, Erik Kawahara, Tatsuya Skantze, Gabriel |
| author_facet | Inoue, Koji Jiang, Bing'er Ekstedt, Erik Kawahara, Tatsuya Skantze, Gabriel |
| contents | A demonstration of a real-time and continuous turn-taking prediction system is presented. The system is based on a voice activity projection (VAP) model, which directly maps dialogue stereo audio to future voice activities. The VAP model includes contrastive predictive coding (CPC) and self-attention transformers, followed by a cross-attention transformer. We examine the effect of the input context audio length and demonstrate that the proposed system can operate in real-time with CPU settings, with minimal performance degradation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_04868 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection Inoue, Koji Jiang, Bing'er Ekstedt, Erik Kawahara, Tatsuya Skantze, Gabriel Computation and Language Human-Computer Interaction Sound Audio and Speech Processing A demonstration of a real-time and continuous turn-taking prediction system is presented. The system is based on a voice activity projection (VAP) model, which directly maps dialogue stereo audio to future voice activities. The VAP model includes contrastive predictive coding (CPC) and self-attention transformers, followed by a cross-attention transformer. We examine the effect of the input context audio length and demonstrate that the proposed system can operate in real-time with CPU settings, with minimal performance degradation. |
| title | Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection |
| topic | Computation and Language Human-Computer Interaction Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2401.04868 |