TAC: Timestamped Audio Captioning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915802564788224 |
|---|---|
| author | Kumar, Sonal Seetharaman, Prem Chen, Ke Nieto, Oriol Su, Jiaqi Wang, Zhepei Kumar, Rithesh Manocha, Dinesh Bryan, Nicholas J. Jin, Zeyu Salamon, Justin |
| author_facet | Kumar, Sonal Seetharaman, Prem Chen, Ke Nieto, Oriol Su, Jiaqi Wang, Zhepei Kumar, Rithesh Manocha, Dinesh Bryan, Nicholas J. Jin, Zeyu Salamon, Justin |
| contents | Large Audio Language Models struggle to disentangle overlapping events in complex acoustic scenes, yielding temporally inconsistent captions and frequent hallucinations. We introduce Timestamped Audio Captioner (TAC), a model that produces temporally grounded audio descriptions at varying degrees of detail and resolution. TAC is trained with a synthetic data pipeline that constructs challenging and dynamic mixtures from real-world audio sources, enabling robust learning under realistic polyphonic conditions. Across event detection and dense captioning, TAC outperforms all competing methods, with a low hallucination rate and accurate temporal grounding. We also introduce TAC-V, an audio-visual pipeline to generate semantically rich audio-visual descriptions. We then show that TAC and TAC-V serves as a "semantic bridge" for a text-only reasoner: a simple TAC$\rightarrow$LLM and TAC-V$\rightarrow$LLM cascade achieves state-of-the-art scores on benchmarks for both audio (MMAU-Pro, MMSU, MMAR) and audio-visual (DailyOmni, VideoHolmes) understanding and reasoning respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15766 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TAC: Timestamped Audio Captioning Kumar, Sonal Seetharaman, Prem Chen, Ke Nieto, Oriol Su, Jiaqi Wang, Zhepei Kumar, Rithesh Manocha, Dinesh Bryan, Nicholas J. Jin, Zeyu Salamon, Justin Sound Large Audio Language Models struggle to disentangle overlapping events in complex acoustic scenes, yielding temporally inconsistent captions and frequent hallucinations. We introduce Timestamped Audio Captioner (TAC), a model that produces temporally grounded audio descriptions at varying degrees of detail and resolution. TAC is trained with a synthetic data pipeline that constructs challenging and dynamic mixtures from real-world audio sources, enabling robust learning under realistic polyphonic conditions. Across event detection and dense captioning, TAC outperforms all competing methods, with a low hallucination rate and accurate temporal grounding. We also introduce TAC-V, an audio-visual pipeline to generate semantically rich audio-visual descriptions. We then show that TAC and TAC-V serves as a "semantic bridge" for a text-only reasoner: a simple TAC$\rightarrow$LLM and TAC-V$\rightarrow$LLM cascade achieves state-of-the-art scores on benchmarks for both audio (MMAU-Pro, MMSU, MMAR) and audio-visual (DailyOmni, VideoHolmes) understanding and reasoning respectively. |
| title | TAC: Timestamped Audio Captioning |
| topic | Sound |
| url | https://arxiv.org/abs/2602.15766 |