The USTC-NERCSLIP Systems for the CHiME-9 MCoRec Challenge
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914363045052416 |
|---|---|
| author | Jiang, Ya Wang, Ruoyu Zhang, Jingxuan Du, Jun Han, Yi Quan, Zihao Chen, Hang Yang, Yeran Zheng, Kongzhi Chen, Zhuo Tu, Yanhui Niu, Shutong Xi, Changfeng Wang, Mengzhi Wu, Zhongbin Chen, Jieru Zhi, Henghui Shi, Weiyi Wu, Shuhang Wan, Genshun Pan, Jia Gao, Jianqing |
| author_facet | Jiang, Ya Wang, Ruoyu Zhang, Jingxuan Du, Jun Han, Yi Quan, Zihao Chen, Hang Yang, Yeran Zheng, Kongzhi Chen, Zhuo Tu, Yanhui Niu, Shutong Xi, Changfeng Wang, Mengzhi Wu, Zhongbin Chen, Jieru Zhi, Henghui Shi, Weiyi Wu, Shuhang Wan, Genshun Pan, Jia Gao, Jianqing |
| contents | This report details our submission to the CHiME-9 MCoRec Challenge on recognizing and clustering multiple concurrent natural conversations within indoor social settings. Unlike conventional meetings centered on a single shared topic, this scenario contains multiple parallel dialogues--up to eight speakers across up to four simultaneous conversations--with a speech overlap rate exceeding 90%. To tackle this, we propose a multimodal cascaded system that leverages per-speaker visual streams extracted from synchronized 360 degree video together with single-channel audio. Our system improves three components of the pipeline by leveraging enhanced audio-visual pretrained models: Active Speaker Detection (ASD), Audio-Visual Target Speech Extraction (AVTSE), and Audio-Visual Speech Recognition (AVSR). The AVSR module further incorporates Whisper and LLM techniques to boost transcription accuracy. Our best single cascaded system achieves a Speaker Word Error Rate (WER) of 32.44% on the development set. By further applying ROVER to fuse outputs from diverse front-end and back-end variants, we reduce Speaker WER to 31.40%. Notably, our LLM-based zero-shot conversational clustering achieves a speaker clustering F1 score of 1.0, yielding a final Joint ASR-Clustering Error Rate (JACER) of 15.70%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_01415 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The USTC-NERCSLIP Systems for the CHiME-9 MCoRec Challenge Jiang, Ya Wang, Ruoyu Zhang, Jingxuan Du, Jun Han, Yi Quan, Zihao Chen, Hang Yang, Yeran Zheng, Kongzhi Chen, Zhuo Tu, Yanhui Niu, Shutong Xi, Changfeng Wang, Mengzhi Wu, Zhongbin Chen, Jieru Zhi, Henghui Shi, Weiyi Wu, Shuhang Wan, Genshun Pan, Jia Gao, Jianqing Audio and Speech Processing This report details our submission to the CHiME-9 MCoRec Challenge on recognizing and clustering multiple concurrent natural conversations within indoor social settings. Unlike conventional meetings centered on a single shared topic, this scenario contains multiple parallel dialogues--up to eight speakers across up to four simultaneous conversations--with a speech overlap rate exceeding 90%. To tackle this, we propose a multimodal cascaded system that leverages per-speaker visual streams extracted from synchronized 360 degree video together with single-channel audio. Our system improves three components of the pipeline by leveraging enhanced audio-visual pretrained models: Active Speaker Detection (ASD), Audio-Visual Target Speech Extraction (AVTSE), and Audio-Visual Speech Recognition (AVSR). The AVSR module further incorporates Whisper and LLM techniques to boost transcription accuracy. Our best single cascaded system achieves a Speaker Word Error Rate (WER) of 32.44% on the development set. By further applying ROVER to fuse outputs from diverse front-end and back-end variants, we reduce Speaker WER to 31.40%. Notably, our LLM-based zero-shot conversational clustering achieves a speaker clustering F1 score of 1.0, yielding a final Joint ASR-Clustering Error Rate (JACER) of 15.70%. |
| title | The USTC-NERCSLIP Systems for the CHiME-9 MCoRec Challenge |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2603.01415 |