Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.21828 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911339286364160 |
|---|---|
| author | Kong, YuXiang Hou, JunFeng Tang, Jian Zhu, Bingqing Zhang, Jicheng Xue, Shaofei |
| author_facet | Kong, YuXiang Hou, JunFeng Tang, Jian Zhu, Bingqing Zhang, Jicheng Xue, Shaofei |
| contents | Large language model (LLM)-based automatic speech recognition (ASR) has recently achieved strong performance across diverse tasks, yet contextual biasing for named entities and hotwords under large vocabularies remains challenging. In this work, we propose a scalable two-stage framework that integrates hotword retrieval with LLM-ASR adaptation. First, we extend the Global-Local Contrastive Language-Audio pre-trained model (GLCLAP) to retrieve a compact top-k set of hotword candidates from a large vocabulary via robustness-aware data augmentation and fuzzy matching. Second, we inject the retrieved candidates as textual prompts into an LLM-ASR model and fine-tune it with Generative Rejection-Based Policy Optimization (GRPO), using a task-driven reward that jointly optimizes hotword recognition and overall transcription accuracy. Experiments on hotword-focused test sets show substantial keyword error rate (KER) reductions while maintaining sentence accuracy on general ASR benchmarks, demonstrating the effectiveness of the proposed framework for large-vocabulary contextual biasing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_21828 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Contextual Biasing for LLM-Based ASR with Hotword Retrieval and Reinforcement Learning Kong, YuXiang Hou, JunFeng Tang, Jian Zhu, Bingqing Zhang, Jicheng Xue, Shaofei Audio and Speech Processing Large language model (LLM)-based automatic speech recognition (ASR) has recently achieved strong performance across diverse tasks, yet contextual biasing for named entities and hotwords under large vocabularies remains challenging. In this work, we propose a scalable two-stage framework that integrates hotword retrieval with LLM-ASR adaptation. First, we extend the Global-Local Contrastive Language-Audio pre-trained model (GLCLAP) to retrieve a compact top-k set of hotword candidates from a large vocabulary via robustness-aware data augmentation and fuzzy matching. Second, we inject the retrieved candidates as textual prompts into an LLM-ASR model and fine-tune it with Generative Rejection-Based Policy Optimization (GRPO), using a task-driven reward that jointly optimizes hotword recognition and overall transcription accuracy. Experiments on hotword-focused test sets show substantial keyword error rate (KER) reductions while maintaining sentence accuracy on general ASR benchmarks, demonstrating the effectiveness of the proposed framework for large-vocabulary contextual biasing. |
| title | Contextual Biasing for LLM-Based ASR with Hotword Retrieval and Reinforcement Learning |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2512.21828 |