Resurfacing Paralinguistic Awareness in Large Audio Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915857445158912 |
|---|---|
| author | Yang, Hao Wang, Minghan Wu, Tongtong Qu, Lizhen Shareghi, Ehsan Haffari, Gholamreza |
| author_facet | Yang, Hao Wang, Minghan Wu, Tongtong Qu, Lizhen Shareghi, Ehsan Haffari, Gholamreza |
| contents | Large Audio Language Models (LALMs) have expanded the interaction with human to speech modality, which introduces great interactive potential, due to the paralinguistic cues implicitly indicating the user context. However, building on the current content-centred paradigm, LALMs usually neglect such paralinguistic cues and respond solely based on query content. In this work, to resurface the paralinguistic awareness in LALMs, we introduce five diverse layer-wise analyses to jointly identify paralinguistic layers and semantic understanding layers. Based on these insights, we propose a paralinguistic-enhanced fine-tuning (PE-FT) protocol accordingly to equip LALMs with paralinguistic-aware capabilities, including (1) selective-layer fine-tuning, and (2) an auxiliary dual-level classification head. Our experiments demonstrate that PE-FT protocol efficiently and effectively resurfaces the paralinguistic awareness, even surpassing the performance of the all-layer fine-tuning strategy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_11947 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Resurfacing Paralinguistic Awareness in Large Audio Language Models Yang, Hao Wang, Minghan Wu, Tongtong Qu, Lizhen Shareghi, Ehsan Haffari, Gholamreza Sound Computation and Language Multimedia Audio and Speech Processing Large Audio Language Models (LALMs) have expanded the interaction with human to speech modality, which introduces great interactive potential, due to the paralinguistic cues implicitly indicating the user context. However, building on the current content-centred paradigm, LALMs usually neglect such paralinguistic cues and respond solely based on query content. In this work, to resurface the paralinguistic awareness in LALMs, we introduce five diverse layer-wise analyses to jointly identify paralinguistic layers and semantic understanding layers. Based on these insights, we propose a paralinguistic-enhanced fine-tuning (PE-FT) protocol accordingly to equip LALMs with paralinguistic-aware capabilities, including (1) selective-layer fine-tuning, and (2) an auxiliary dual-level classification head. Our experiments demonstrate that PE-FT protocol efficiently and effectively resurfaces the paralinguistic awareness, even surpassing the performance of the all-layer fine-tuning strategy. |
| title | Resurfacing Paralinguistic Awareness in Large Audio Language Models |
| topic | Sound Computation and Language Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2603.11947 |