Cover Image

MUSIC: MUlti-Step Instruction Contrast for Multi-Turn Reward Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Wenzhe, Zhang, Shujian, Zhou, Wenxuan, Lambert, John, Jin, Chi, Hard, Andrew, Mathews, Rajiv, Wang, Lun
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Eliciting Behaviors in Multi-Turn Conversations
by: Huang, Jing, et al.
Published: (2025)

Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process
by: Zhang, Zhenyu, et al.
Published: (2025)

Steering LLMs for Culturally Localized Generation
by: Khanuja, Simran, et al.
Published: (2026)

DPO Unchained: Your Training Algorithm is Secretly Disentangled in Human Choice Theory
by: Zhou, Wenxuan, et al.
Published: (2025)

T-REG: Preference Optimization with Token-Level Reward Regularization
by: Zhou, Wenxuan, et al.
Published: (2024)

SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe
by: Xiao, Yuxin, et al.
Published: (2024)

DRUM: Learning Demonstration Retriever for Large MUlti-modal Models
by: Yi-Ge, Ellen, et al.
Published: (2024)

DLM-One: Diffusion Language Models for One-Step Sequence Generation
by: Chen, Tianqi, et al.
Published: (2025)

Multi-Turn Code Generation Through Single-Step Rewards
by: Jain, Arnav Kumar, et al.
Published: (2025)

TurnBench-MS: A Benchmark for Evaluating Multi-Turn, Multi-Step Reasoning in Large Language Models
by: Zhang, Yiran, et al.
Published: (2025)

Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models
by: Sun, Yuchong, et al.
Published: (2023)

Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following
by: He, Yun, et al.
Published: (2024)

GRRM: Group Relative Reward Modeling for Machine Translation
by: Yang, Sen, et al.
Published: (2026)

Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions
by: Li, Jiahuan, et al.
Published: (2023)

Contrastive Instruction Tuning
by: Yan, Tianyi Lorena, et al.
Published: (2024)

Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
by: Li, Wenzhe, et al.
Published: (2025)

Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy
by: Wu, Tong, et al.
Published: (2024)

Process-based Self-Rewarding Language Models
by: Zhang, Shimao, et al.
Published: (2025)

Can Language Models Follow Multiple Turns of Entangled Instructions?
by: Han, Chi, et al.
Published: (2025)

Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward
by: Wan, Yanming, et al.
Published: (2025)

Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues
by: Ou, Jiao, et al.
Published: (2024)

R-PRM: Reasoning-Driven Process Reward Modeling
by: She, Shuaijie, et al.
Published: (2025)

TurnWise: The Gap between Single- and Multi-turn Language Model Capabilities
by: Graf, Victoria, et al.
Published: (2026)

Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards
by: Yuan, Youliang, et al.
Published: (2025)

TAMTRL: Teacher-Aligned Reward Reshaping for Multi-Turn Reinforcement Learning in Long-Context Compression
by: Wang, Li, et al.
Published: (2026)

Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models
by: Wang, Teng, et al.
Published: (2025)

GraphIF: Enhancing Multi-Turn Instruction Following for Large Language Models with Relation Graph Prompt
by: Li, Zhenhe, et al.
Published: (2025)

Stop Rewarding Hallucinated Steps: Faithfulness-Aware Step-Level Reinforcement Learning for Small Reasoning Models
by: Nie, Shuo, et al.
Published: (2026)

RewardBench 2: Advancing Reward Model Evaluation
by: Malik, Saumya, et al.
Published: (2025)

Agentic Reinforcement Learning with Implicit Step Rewards
by: Liu, Xiaoqian, et al.
Published: (2025)

MUSER: A MUlti-Step Evidence Retrieval Enhancement Framework for Fake News Detection
by: Liao, Hao, et al.
Published: (2023)

Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards
by: Jiayang, Cheng, et al.
Published: (2026)

CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdom
by: Li, Yisen, et al.
Published: (2025)

IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation
by: Wen, Bosi, et al.
Published: (2026)

MRO: Enhancing Reasoning in Diffusion Language Models via Multi-Reward Optimization
by: Wang, Chenglong, et al.
Published: (2025)

On the Multi-turn Instruction Following for Conversational Web Agents
by: Deng, Yang, et al.
Published: (2024)

AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress
by: Xi, Zhiheng, et al.
Published: (2025)

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning
by: Xu, Huimin, et al.
Published: (2025)

Training Data Efficiency in Multimodal Process Reward Models
by: Li, Jinyuan, et al.
Published: (2026)

Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewards
by: Pisano, Raffaele, et al.
Published: (2026)