Cover Image

What do MLLMs hear? Examining reasoning with text and sound components in Multimodal Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Çoban, Enis Berk, Mandel, Michael I., Devaney, Johanna
Format:	Preprint
Published:	2024
Subjects:	Audio and Speech Processing Computation and Language Sound
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Some clues to build a sound analysis relevant to hearing
by: Millot, Laurent
Published: (2024)

pyAMPACT: A Score-Audio Alignment Toolkit for Performance Data Estimation and Multi-modal Processing
by: Devaney, Johanna, et al.
Published: (2024)

Investigating Decoder-only Large Language Models for Speech-to-text Translation
by: Huang, Chao-Wei, et al.
Published: (2024)

Signal processing algorithm effective for sound quality of hearing loss simulators
by: Irino, Toshio, et al.
Published: (2024)

SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Models
by: Peri, Raghuveer, et al.
Published: (2024)

WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning
by: Mundada, Gagan, et al.
Published: (2025)

CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models
by: Wu, Shangda, et al.
Published: (2024)

Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samples
by: Kuan, Chun-Yi, et al.
Published: (2025)

AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation
by: Manakul, Potsawee, et al.
Published: (2025)

On the relationship between speech and hearing
by: Umesh, Srinivasan, et al.
Published: (2024)

$\text{M}^3\text{PDB}$: A Multimodal, Multi-Label, Multilingual Prompt Database for Speech Generation
by: Zhu, Boyu, et al.
Published: (2025)

An efficient text augmentation approach for contextualized Mandarin speech recognition
by: Zheng, Naijun, et al.
Published: (2024)

Transferable speech-to-text large language model alignment module
by: Wu, Boyong, et al.
Published: (2024)

Fine-Tuning Large Multimodal Models for Automatic Pronunciation Assessment
by: Wang, Ke, et al.
Published: (2025)

GLAP: General contrastive audio-text pretraining across domains and languages
by: Dinkel, Heinrich, et al.
Published: (2025)

Natural language guidance of high-fidelity text-to-speech with synthetic annotations
by: Lyth, Dan, et al.
Published: (2024)

Exploring the Potential of Large Multimodal Models as Effective Alternatives for Pronunciation Assessment
by: Wang, Ke, et al.
Published: (2025)

Data-driven grapheme-to-phoneme representations for a lexicon-free text-to-speech
by: Garg, Abhinav, et al.
Published: (2024)

High-precision Voice Search Query Correction via Retrievable Speech-text Embedings
by: Li, Christopher, et al.
Published: (2024)

Strategies for improving low resource speech to text translation relying on pre-trained ASR models
by: Kesiraju, Santosh, et al.
Published: (2023)

Synthetic training set generation using text-to-audio models for environmental sound classification
by: Ronchini, Francesca, et al.
Published: (2024)

Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights
by: Yang, Hao, et al.
Published: (2024)

ALAS: Measuring Latent Speech-Text Alignment For Spoken Language Understanding In Multimodal LLMs
by: Mousavi, Pooneh, et al.
Published: (2025)

Large Language Models for Dysfluency Detection in Stuttered Speech
by: Wagner, Dominik, et al.
Published: (2024)

ASR Error Correction using Large Language Models
by: Ma, Rao, et al.
Published: (2024)

How Contrastive Decoding Enhances Large Audio Language Models?
by: Lin, Tzu-Quan, et al.
Published: (2026)

BLSP-Emo: Towards Empathetic Large Speech-Language Models
by: Wang, Chen, et al.
Published: (2024)

An End-to-End Speech Summarization Using Large Language Model
by: Shang, Hengchao, et al.
Published: (2024)

Closing the Modality Reasoning Gap for Speech Large Language Models
by: Wang, Chaoren, et al.
Published: (2026)

Customizing Speech Recognition Model with Large Language Model Feedback
by: Ling, Shaoshi, et al.
Published: (2025)

Spatial Audio Processing with Large Language Model on Wearable Devices
by: Mishra, Ayushi, et al.
Published: (2025)

Boosting Large Language Model for Speech Synthesis: An Empirical Study
by: Hao, Hongkun, et al.
Published: (2023)

SALMONN: Towards Generic Hearing Abilities for Large Language Models
by: Tang, Changli, et al.
Published: (2023)

Direct Simultaneous Translation Activation for Large Audio-Language Models
by: Zhang, Pei, et al.
Published: (2025)

Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speech
by: Kang, Wonjune, et al.
Published: (2024)

ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models
by: Li, Bohan, et al.
Published: (2025)

Leveraging Large Language Models for Sarcastic Speech Annotation in Sarcasm Detection
by: Li, Zhu, et al.
Published: (2025)

Audio Large Language Models Can Be Descriptive Speech Quality Evaluators
by: Chen, Chen, et al.
Published: (2025)

Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversation
by: Kim, Heeseung, et al.
Published: (2024)

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
by: Zhang, Xin, et al.
Published: (2023)