Cover Image

MTalk-Bench: Evaluating Speech-to-Speech Models in Multi-Turn Dialogues via Arena-style and Rubrics Protocols

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Du, Yuhao, Huang, Qianwei, Zhu, Guo, Dai, Zhanchen, Chen, Shunian, Zhu, Qiming, Pan, Le, Chen, Minghao, Zhang, Yuhao, Zhou, Li, Wang, Benyou, Li, Haizhou
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

EchoX: Towards Mitigating Acoustic-Semantic Gap via Echo Training for Speech-to-Speech LLMs
by: Zhang, Yuhao, et al.
Published: (2025)

S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models
by: Jiang, Feng, et al.
Published: (2025)

Ti-Audio: The First Multi-Dialectal End-to-End Speech LLM for Tibetan
by: Wang, Jialing, et al.
Published: (2026)

Soundwave: Less is More for Speech-Text Alignment in LLMs
by: Zhang, Yuhao, et al.
Published: (2025)

Roadmap towards Superhuman Speech Understanding using Large Language Models
by: Bu, Fan, et al.
Published: (2024)

From Lossy to Verified: A Provenance-Aware Tiered Memory for Agents
by: Zhu, Qiming, et al.
Published: (2026)

EchoMind: An Interrelated Multi-level Benchmark for Evaluating Empathetic Speech Language Models
by: Zhou, Li, et al.
Published: (2025)

Leveraging Unit Language Guidance to Advance Speech Modeling in Textless Speech-to-Speech Translation
by: Zhang, Yuhao, et al.
Published: (2025)

BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement
by: Du, Yuhao, et al.
Published: (2024)

CS3-Bench: Evaluating and Enhancing Speech-to-Speech LLMs for Mandarin-English Code-Switching
by: Liu, Heyang, et al.
Published: (2025)

EvA: An Evidence-First Audio Understanding Paradigm for LALMs
by: Xie, Xinyuan, et al.
Published: (2026)

MileBench: Benchmarking MLLMs in Long Context
by: Song, Dingjie, et al.
Published: (2024)

From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test
by: Dai, Xunlian, et al.
Published: (2025)

VocalBench-DF: A Benchmark for Evaluating Speech LLM Robustness to Disfluency
by: Liu, Hongcheng, et al.
Published: (2025)

Retrieval-Augmented Dialogue Knowledge Aggregation for Expressive Conversational Speech Synthesis
by: Liu, Rui, et al.
Published: (2025)

MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
by: Ge, Wentao, et al.
Published: (2023)

Training-Free Text-to-Image Compositional Food Generation via Prompt Grafting
by: Pan, Xinyue, et al.
Published: (2026)

Exploring Length Generalization For Transformer-based Speech Enhancement
by: Zhang, Qiquan, et al.
Published: (2025)

An Exploration of Length Generalization in Transformer-Based Speech Enhancement
by: Zhang, Qiquan, et al.
Published: (2024)

Synchronization and Turn-Taking in Full-Duplex Speech Dialogue Models
by: Riera, Pablo, et al.
Published: (2026)

VocalBench-zh: Decomposing and Benchmarking the Speech Conversational Abilities in Mandarin Context
by: Liu, Heyang, et al.
Published: (2025)

VocalBench: Benchmarking the Vocal Conversational Abilities for Speech Interaction Models
by: Liu, Heyang, et al.
Published: (2025)

Automated Rubrics for Reliable Evaluation of Medical Dialogue Systems
by: Chen, Yinzhu, et al.
Published: (2026)

Speech Separation with Pretrained Frontend to Minimize Domain Mismatch
by: Wang, Wupeng, et al.
Published: (2024)

Enhancing Speech-to-Speech Dialogue Modeling with End-to-End Retrieval-Augmented Generation
by: Feng, Pengchao, et al.
Published: (2025)

X-Talk: On the Underestimated Potential of Modular Speech-to-Speech Dialogue System
by: Liu, Zhanxun, et al.
Published: (2025)

Target Speech Diarization with Multimodal Prompts
by: Jiang, Yidi, et al.
Published: (2024)

Humans or LLMs as the Judge? A Study on Judgement Biases
by: Chen, Guiming Hardy, et al.
Published: (2024)

SpeechColab Leaderboard: An Open-Source Platform for Automatic Speech Recognition Evaluation
by: Du, Jiayu, et al.
Published: (2024)

Direct Preference Optimization for Speech Autoregressive Diffusion Models
by: Liu, Zhijun, et al.
Published: (2025)

Causal Self-supervised Pretrained Frontend with Predictive Code for Speech Separation
by: Wang, Wupeng, et al.
Published: (2025)

SOVA-Bench: Benchmarking the Speech Conversation Ability for LLM-based Voice Assistant
by: Hou, Yixuan, et al.
Published: (2025)

Coded Beam Training for RIS Assisted Wireless Communications
by: Chen, Yuhao, et al.
Published: (2024)

Prompt-driven Target Speech Diarization
by: Jiang, Yidi, et al.
Published: (2023)

A Comprehensive Rubric for Annotating Pathological Speech
by: Corrales-Astorgano, Mario, et al.
Published: (2024)

PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation
by: Chen, Xin-Sheng, et al.
Published: (2026)

Computation and Parameter Efficient Multi-Modal Fusion Transformer for Cued Speech Recognition
by: Liu, Lei, et al.
Published: (2024)

KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues
by: Chen, Junhao, et al.
Published: (2025)

DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Report
by: Li, Ruizhe, et al.
Published: (2026)

AffectSpeech: A Large-Scale Emotional Speech Dataset with Fine-Grained Textual Descriptions for Speech Emotion Captioning and Synthesis
by: Qi, Tianhua, et al.
Published: (2026)