Cover Image

IFEval-Audio: Benchmarking Instruction-Following Capability in Audio-based Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gao, Yiming, Wang, Bin, Wei, Chengwei, Sun, Shuo, Aw, AiTi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AudioBench: A Universal Benchmark for Audio Large Language Models
by: Wang, Bin, et al.
Published: (2024)

MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models
by: He, Yingxu, et al.
Published: (2024)

Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models
by: Wang, Bin, et al.
Published: (2025)

MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders
by: Zhang, Wenyu, et al.
Published: (2024)

WildIFEval: Instruction Following in the Wild
by: Lior, Gili, et al.
Published: (2025)

IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languages
by: Jayakumar, Thanmay, et al.
Published: (2026)

SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning
by: Wang, Bin, et al.
Published: (2023)

Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs
by: Quang, Trung Nguyen, et al.
Published: (2026)

M-IFEval: Multilingual Instruction-Following Evaluation
by: Dussolle, Antoine, et al.
Published: (2025)

Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models
by: Manakul, Potsawee, et al.
Published: (2024)

CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation
by: Zheng, Weihua, et al.
Published: (2025)

Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Data
by: Wang, Qiongqiong, et al.
Published: (2025)

Improving Multilingual Social Media Insights: Aspect-based Comment Analysis
by: Zhang, Longyin, et al.
Published: (2025)

Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?
by: Zhang, Qinyan, et al.
Published: (2025)

Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
by: Lu, Ke-Han, et al.
Published: (2025)

PALM-Bench: A Comprehensive Benchmark for Personalized Audio-Language Models
by: Wang, Yuwen, et al.
Published: (2026)

Beyond Classification: Towards Speech Emotion Reasoning with Multitask AudioLLMs
by: Zhang, Wenyu, et al.
Published: (2025)

Resilience of Large Language Models for Noisy Instructions
by: Wang, Bin, et al.
Published: (2024)

ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models
by: Li, Bohan, et al.
Published: (2025)

Words at Play: Benchmarking Audio Pun Understanding in Large Audio-Language Models
by: Su, Yuchen, et al.
Published: (2026)

AudioTrust: Benchmarking the Multifaceted Trustworthiness of Audio Large Language Models
by: Li, Kai, et al.
Published: (2025)

Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models
by: Chen, Yiming, et al.
Published: (2024)

MERaLiON-TextLLM: Cross-Lingual Understanding of Large Language Models in Chinese, Indonesian, Malay, and Singlish
by: Huang, Xin, et al.
Published: (2024)

Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models
by: Gao, Kuofeng, et al.
Published: (2024)

ALLM4ADD: Unlocking the Capabilities of Audio Large Language Models for Audio Deepfake Detection
by: Gu, Hao, et al.
Published: (2025)

Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models
by: Xie, Zhifei, et al.
Published: (2025)

AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios
by: Qi, Yunjia, et al.
Published: (2025)

Incorporating Contextual Paralinguistic Understanding in Large Speech-Language Models
by: Wang, Qiongqiong, et al.
Published: (2025)

Causal Tracing of Audio-Text Fusion in Large Audio Language Models
by: Chen, Wei-Chih, et al.
Published: (2026)

Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language Models
by: Zeng, Bo, et al.
Published: (2025)

EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Models
by: Zou, Tao, et al.
Published: (2025)

IFEvalCode: Controlled Code Generation
by: Yang, Jian, et al.
Published: (2025)

Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding
by: Liu, Jizhong, et al.
Published: (2024)

Harnessing Multi-Role Capabilities of Large Language Models for Open-Domain Question Answering
by: Sun, Hongda, et al.
Published: (2024)

SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models
by: Yang, Wanqi, et al.
Published: (2025)

Direct Simultaneous Translation Activation for Large Audio-Language Models
by: Zhang, Pei, et al.
Published: (2025)

Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Study
by: Hou, Guanyu, et al.
Published: (2025)

Instruction-Following Pruning for Large Language Models
by: Hou, Bairu, et al.
Published: (2025)

SeaLLMs-Audio: Large Audio-Language Models for Southeast Asia
by: Liu, Chaoqun, et al.
Published: (2025)

VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents
by: Hu, Jiliang, et al.
Published: (2025)