End-to-end audio-visual learning for cochlear implant sound coding simulations in noisy environments
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915760209657856 |
|---|---|
| author | Lin, Meng-Ping Huang, Enoch Hsin-Ho Chien, Shao-Yi Tsao, Yu |
| author_facet | Lin, Meng-Ping Huang, Enoch Hsin-Ho Chien, Shao-Yi Tsao, Yu |
| contents | The cochlear implant (CI) is a successful biomedical device that enables individuals with severe-to-profound hearing loss to perceive sound through electrical stimulation, yet listening in noise remains challenging. Recent deep learning advances offer promising potential for CI sound coding by integrating visual cues. In this study, an audio-visual speech enhancement (AVSE) module is integrated with the ElectrodeNet-CS (ECS) model to form the end-to-end CI system, AVSE-ECS. Simulations show that the AVSE-ECS system with joint training achieves high objective speech intelligibility and improves the signal-to-error ratio (SER) by 7.4666 dB compared to the advanced combination encoder (ACE) strategy. These findings underscore the potential of AVSE-based CI sound coding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_13576 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | End-to-end audio-visual learning for cochlear implant sound coding simulations in noisy environments Lin, Meng-Ping Huang, Enoch Hsin-Ho Chien, Shao-Yi Tsao, Yu Audio and Speech Processing Artificial Intelligence Sound Image and Video Processing The cochlear implant (CI) is a successful biomedical device that enables individuals with severe-to-profound hearing loss to perceive sound through electrical stimulation, yet listening in noise remains challenging. Recent deep learning advances offer promising potential for CI sound coding by integrating visual cues. In this study, an audio-visual speech enhancement (AVSE) module is integrated with the ElectrodeNet-CS (ECS) model to form the end-to-end CI system, AVSE-ECS. Simulations show that the AVSE-ECS system with joint training achieves high objective speech intelligibility and improves the signal-to-error ratio (SER) by 7.4666 dB compared to the advanced combination encoder (ACE) strategy. These findings underscore the potential of AVSE-based CI sound coding. |
| title | End-to-end audio-visual learning for cochlear implant sound coding simulations in noisy environments |
| topic | Audio and Speech Processing Artificial Intelligence Sound Image and Video Processing |
| url | https://arxiv.org/abs/2508.13576 |