Two-stage Audio-Visual Target Speaker Extraction System for Real-Time Processing On Edge Device
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914152092532736 |
|---|---|
| author | Li, Zixuan Zhang, Xueliang Miao, Lei Yan, Zhipeng Sun, Ying Zhu, Chong |
| author_facet | Li, Zixuan Zhang, Xueliang Miao, Lei Yan, Zhipeng Sun, Ying Zhu, Chong |
| contents | Audio-Visual Target Speaker Extraction (AVTSE) aims to isolate a target speaker's voice in a multi-speaker environment with visual cues as auxiliary. Most of the existing AVTSE methods encode visual and audio features simultaneously, resulting in extremely high computational complexity and making it impractical for real-time processing on edge devices. To tackle this issue, we proposed a two-stage ultra-compact AVTSE system. Specifically, in the first stage, a compact network is employed for voice activity detection (VAD) using visual information. In the second stage, the VAD results are combined with audio inputs to isolate the target speaker's voice. Experiments show that the proposed system effectively suppresses background noise and interfering voices while spending little computational resources. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_22229 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Two-stage Audio-Visual Target Speaker Extraction System for Real-Time Processing On Edge Device Li, Zixuan Zhang, Xueliang Miao, Lei Yan, Zhipeng Sun, Ying Zhu, Chong Sound Audio and Speech Processing Audio-Visual Target Speaker Extraction (AVTSE) aims to isolate a target speaker's voice in a multi-speaker environment with visual cues as auxiliary. Most of the existing AVTSE methods encode visual and audio features simultaneously, resulting in extremely high computational complexity and making it impractical for real-time processing on edge devices. To tackle this issue, we proposed a two-stage ultra-compact AVTSE system. Specifically, in the first stage, a compact network is employed for voice activity detection (VAD) using visual information. In the second stage, the VAD results are combined with audio inputs to isolate the target speaker's voice. Experiments show that the proposed system effectively suppresses background noise and interfering voices while spending little computational resources. |
| title | Two-stage Audio-Visual Target Speaker Extraction System for Real-Time Processing On Edge Device |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.22229 |