Cover Image

Mutual-Taught for Co-adapting Policy and Reward Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shi, Tianyuan, Huang, Canbin, Wan, Fanqi, Zhong, Longguang, Yang, Ziyi, Shen, Weizhou, Quan, Xiaojun, Yan, Ming
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Weighted-Reward Preference Optimization for Implicit Model Fusion
by: Yang, Ziyi, et al.
Published: (2024)

ProFuser: Progressive Fusion of Large Language Models
by: Shi, Tianyuan, et al.
Published: (2024)

FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion
by: Yang, Ziyi, et al.
Published: (2025)

FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion
by: Zhong, Longguang, et al.
Published: (2025)

FuseChat: Knowledge Fusion of Chat Models
by: Wan, Fanqi, et al.
Published: (2024)

Knowledge Fusion of Chat LLMs: A Preliminary Technical Report
by: Wan, Fanqi, et al.
Published: (2024)

SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models
by: Yang, Ziyi, et al.
Published: (2025)

ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents
by: Liu, Tianjian, et al.
Published: (2025)

Small LLMs Are Weak Tool Learners: A Multi-LLM Agent
by: Shen, Weizhou, et al.
Published: (2024)

BlockPruner: Fine-grained Pruning for Large Language Models
by: Zhong, Longguang, et al.
Published: (2024)

SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization
by: Sun, Huashan, et al.
Published: (2025)

Lookahead Routing for Large Language Models
by: Huang, Canbin, et al.
Published: (2025)

CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning
by: Lu, Zhiyuan, et al.
Published: (2026)

Co-Evolution of Policy and Internal Reward for Language Agents
by: Wang, Xinyu, et al.
Published: (2026)

Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models
by: Hong, Haitao, et al.
Published: (2025)

DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
by: Li, Gang, et al.
Published: (2025)

ThinkSwitcher: When to Think Hard, When to Think Fast
by: Liang, Guosheng, et al.
Published: (2025)

Evidence-Augmented Policy Optimization with Reward Co-Evolution for Long-Context Reasoning
by: Guan, Xin, et al.
Published: (2026)

Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning
by: Chen, Yifang, et al.
Published: (2024)

Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks
by: Shen, Huanming, et al.
Published: (2025)

Self-Taught Evaluators
by: Wang, Tianlu, et al.
Published: (2024)

Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shaping
by: Peng, Miao, et al.
Published: (2026)

PaCoST: Paired Confidence Significance Testing for Benchmark Contamination Detection in Large Language Models
by: Zhang, Huixuan, et al.
Published: (2024)

Rényi Divergence Deep Mutual Learning
by: Huang, Weipeng, et al.
Published: (2022)

Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector
by: Yang, Haihui, et al.
Published: (2024)

Policy Filtration for RLHF to Mitigate Noise in Reward Models
by: Zhang, Chuheng, et al.
Published: (2024)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
by: Lu, Taiming, et al.
Published: (2024)

Efficient Process Reward Modeling via Contrastive Mutual Information
by: Lee, Nakyung, et al.
Published: (2026)

Self-Taught Agentic Long Context Understanding
by: Zhuang, Yufan, et al.
Published: (2025)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
by: Ye, Ziyi, et al.
Published: (2024)

Secrets of RLHF in Large Language Models Part II: Reward Modeling
by: Wang, Binghai, et al.
Published: (2024)

Fine-Tuning Language Models with Reward Learning on Policy
by: Lang, Hao, et al.
Published: (2024)

InfoPO: Information-Driven Policy Optimization for User-Centric Agents
by: Kong, Fanqi, et al.
Published: (2026)

Telogenesis: Goal Is All U Need
by: Deng, Zhuoran, et al.
Published: (2026)

Reward Bound for Behavioral Guarantee of Model-based Planning Agents
by: An, Zhiyu, et al.
Published: (2024)

Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
by: Yin, Yueqin, et al.
Published: (2025)

CROP: Conservative Reward for Model-based Offline Policy Optimization
by: Li, Hao, et al.
Published: (2023)

Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation
by: Zhang, Xiaoying, et al.
Published: (2024)

Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs
by: Liu, Chris Yuhao, et al.
Published: (2024)

QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
by: Wan, Fanqi, et al.
Published: (2025)