Cover Image

From Demonstrations to Rewards: Alignment Without Explicit Human Preferences

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Siliang, Liu, Yao, Rangwala, Huzefa, Karypis, George, Hong, Mingyi, Fakoor, Rasool
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokens
by: Cen, Zhepeng, et al.
Published: (2024)

Offline Learning and Forgetting for Reasoning with Large Language Models
by: Ni, Tianwei, et al.
Published: (2025)

AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents
by: Yang, Ke, et al.
Published: (2024)

Budgeting Counterfactual for Offline RL
by: Liu, Yao, et al.
Published: (2023)

When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning
by: Zeng, Siliang, et al.
Published: (2023)

Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guarantees
by: Zeng, Siliang, et al.
Published: (2022)

Learning Reward and Policy Jointly from Demonstration and Preference Improves Alignment
by: Li, Chenliang, et al.
Published: (2024)

Learning the Target Network in Function Space
by: Asadi, Kavosh, et al.
Published: (2024)

Scalable Prompt Routing via Fine-Grained Latent Task Discovery
by: Zhang, Yunyi, et al.
Published: (2026)

Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
by: Li, Jiaxiang, et al.
Published: (2024)

Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training
by: Fakoor, Rasool, et al.
Published: (2026)

Protein Structure Tokenization: Benchmarking and New Recipe
by: Yuan, Xinyu, et al.
Published: (2025)

TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
by: Liu, Zuxin, et al.
Published: (2023)

EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
by: Zhang, Jesse, et al.
Published: (2024)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
by: Zhang, Shenao, et al.
Published: (2024)

MaxCode: A Max-Reward Reinforcement Learning Framework for Automated Code Optimization
by: Ou, Jiefu, et al.
Published: (2026)

Reward Learning From Preference With Ties
by: Liu, Jinsong, et al.
Published: (2024)

The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?
by: Pengmei, Zihan, et al.
Published: (2025)

GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning
by: Mavromatis, Costas, et al.
Published: (2024)

ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Models
by: He, Andre, et al.
Published: (2026)

The Alignment Tax: Response Homogenization in Aligned LLMs and Its Implications for Uncertainty Estimation
by: Liu, Mingyi
Published: (2026)

Capturing Individual Human Preferences with Reward Features
by: Barreto, André, et al.
Published: (2025)

Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modeling
by: Xie, Lipeng, et al.
Published: (2025)

Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning
by: Rajaram, Sara, et al.
Published: (2025)

From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
by: Gumbsch, Christian, et al.
Published: (2026)

HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents
by: Peng, Jiangweizhi, et al.
Published: (2026)

PREFINE: Preference-Based Implicit Reward and Cost Fine-Tuning for Safety Alignment
by: Verma, Richa, et al.
Published: (2026)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
by: Huang, Kexin, et al.
Published: (2025)

When Does Multimodality Lead to Better Time Series Forecasting?
by: Zhang, Xiyuan, et al.
Published: (2025)

Dynamic Policy Fusion for User Alignment Without Re-Interaction
by: Palattuparambil, Ajsal Shereef, et al.
Published: (2024)

Chronos-2: From Univariate to Universal Forecasting
by: Ansari, Abdul Fatir, et al.
Published: (2025)

A Unified Pair-GRPO Family: From Implicit to Explicit Preference Constraints for Stable and General RL Alignment
by: Yu, Hao
Published: (2026)

MLZero: A Multi-Agent System for End-to-end Machine Learning Automation
by: Fang, Haoyang, et al.
Published: (2025)

PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model
by: Lin, Baijiong, et al.
Published: (2025)

Human Alignment of Large Language Models through Online Preference Optimisation
by: Calandriello, Daniele, et al.
Published: (2024)

Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
by: Liu, Chris Yuhao, et al.
Published: (2025)

Batch Active Learning of Reward Functions from Human Preferences
by: Bıyık, Erdem, et al.
Published: (2024)

A Unified Linear Programming Framework for Offline Reward Learning from Human Demonstrations and Feedback
by: Kim, Kihyun, et al.
Published: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

AutoG: Towards automatic graph construction from tabular data
by: Chen, Zhikai, et al.
Published: (2025)