Imagen de Portada

Limited Preference Data? Learning Better Reward Model with Latent Space Synthesis

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Tao, Leitian, Du, Xuefeng, Li, Sharon
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense
por: Tao, Leitian, et al.
Publicado: (2025)

Your Weak LLM is Secretly a Strong Teacher for Alignment
por: Tao, Leitian, et al.
Publicado: (2024)

Challenges and Future Directions of Data-Centric AI Alignment
por: Yeh, Min-Hsuan, et al.
Publicado: (2024)

CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement
por: Tao, Leitian, et al.
Publicado: (2024)

Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model
por: Gou, Qi, et al.
Publicado: (2024)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
por: Qiu, Wenjie, et al.
Publicado: (2025)

Efficient Safety Alignment of Large Language Models via Preference Re-ranking and Representation-based Reward Modeling
por: Deng, Qiyuan, et al.
Publicado: (2025)

Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughts
por: Du, Hanwen, et al.
Publicado: (2025)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
por: Ye, Ziyi, et al.
Publicado: (2024)

Learning Ordinal Probabilistic Reward from Preferences
por: Chen, Longze, et al.
Publicado: (2026)

Better Process Supervision with Bi-directional Rewarding Signals
por: Chen, Wenxiang, et al.
Publicado: (2025)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
por: Lin, Yong, et al.
Publicado: (2024)

Generalizing Reward Modeling for Out-of-Distribution Preference Learning
por: Jia, Chen
Publicado: (2024)

Boosting Reward Model with Preference-Conditional Multi-Aspect Synthetic Data Generation
por: Shen, Jiaming, et al.
Publicado: (2024)

Checklists Are Better Than Reward Models For Aligning Language Models
por: Viswanathan, Vijay, et al.
Publicado: (2025)

Shallow Preference Signals: Large Language Model Aligns Even Better with Truncated Data?
por: Qi, Xuan, et al.
Publicado: (2025)

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis
por: Kong, Zicheng, et al.
Publicado: (2026)

Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment
por: Yeh, Samuel, et al.
Publicado: (2025)

Preference Poisoning Attacks on Reward Model Learning
por: Wu, Junlin, et al.
Publicado: (2024)

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data
por: Guo, Siqi, et al.
Publicado: (2025)

A Survey on Human Preference Learning for Large Language Models
por: Jiang, Ruili, et al.
Publicado: (2024)

Remedy: Learning Machine Translation Evaluation from Human Preferences with Reward Modeling
por: Tan, Shaomu, et al.
Publicado: (2025)

Bayesian Preference Learning for Test-Time Steerable Reward Models
por: Hong, Jiwoo, et al.
Publicado: (2026)

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data
por: Wang, Chenglong, et al.
Publicado: (2024)

Weighted-Reward Preference Optimization for Implicit Model Fusion
por: Yang, Ziyi, et al.
Publicado: (2024)

ImplicitRM: Unbiased Reward Modeling from Implicit Preference Data for LLM alignment
por: Wang, Hao, et al.
Publicado: (2026)

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models
por: Chen, Yanjun, et al.
Publicado: (2024)

EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferences
por: Ghate, Kshitish, et al.
Publicado: (2025)

VRM: Teaching Reward Models to Understand Authentic Human Preferences
por: Liu, Biao, et al.
Publicado: (2026)

Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
por: Peng, Andi, et al.
Publicado: (2024)

Steer LLM Latents for Hallucination Detection
por: Park, Seongheon, et al.
Publicado: (2025)

Better Language Model-Based Judging Reward Modeling through Scaling Comprehension Boundaries
por: Ning, Meiling, et al.
Publicado: (2025)

Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems
por: Peng, Hao, et al.
Publicado: (2025)

Better, Faster: Harnessing Self-Improvement in Large Reasoning Models
por: Zhong, Qihuang, et al.
Publicado: (2026)

Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignment
por: Zhang, Jiazheng, et al.
Publicado: (2025)

Gradient-Regularized Latent Space Modulation in Large Language Models for Structured Contextual Synthesis
por: Yotheringhay, Derek, et al.
Publicado: (2025)

Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
por: Gong, Zhuocheng, et al.
Publicado: (2025)

Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?
por: Choi, Hyeong Kyu, et al.
Publicado: (2025)

Distribution-Aware Reward Estimation for Test-Time Reinforcement Learning
por: Du, Bodong, et al.
Publicado: (2026)

Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
por: Jin, Zhuoran, et al.
Publicado: (2025)