Image de couverture de livre

Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhao, Shuai, Xu, Yunqiu, Zhu, Linchao, Yang, Yi
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

RAVR: Reference-Answer-guided Variational Reasoning for Large Language Models
par: Lin, Tianqianjin, et autres
Publié: (2025)

Self-Play Preference Optimization for Language Model Alignment
par: Wu, Yue, et autres
Publié: (2024)

Group Preference Optimization: Few-Shot Alignment of Large Language Models
par: Zhao, Siyan, et autres
Publié: (2023)

Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment
par: Zhang, Yifan, et autres
Publié: (2024)

Binary Classifier Optimization for Large Language Model Alignment
par: Jung, Seungjae, et autres
Publié: (2024)

Accelerated Preference Optimization for Large Language Model Alignment
par: He, Jiafan, et autres
Publié: (2024)

HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages
par: Wang, Zhilin, et autres
Publié: (2025)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
par: Wu, Junkang, et autres
Publié: (2024)

Measuring and Reducing LLM Hallucination without Gold-Standard Answers
par: Wei, Jiaheng, et autres
Publié: (2024)

Alignment through Meta-Weighted Online Sampling: Bridging the Gap between Data Generation and Preference Optimization
par: Yang, Junming, et autres
Publié: (2025)

LifeAlign: Lifelong Alignment for Large Language Models with Memory-Augmented Focalized Preference Optimization
par: Li, Junsong, et autres
Publié: (2025)

Rewarding Intellectual Humility Learning When Not To Answer In Large Language Models
par: Jha, Abha, et autres
Publié: (2026)

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference
par: Gao, Mingqi, et autres
Publié: (2024)

MaxMin-RLHF: Alignment with Diverse Human Preferences
par: Chakraborty, Souradip, et autres
Publié: (2024)

ProofOptimizer: Training Language Models to Simplify Proofs without Human Demonstrations
par: Gu, Alex, et autres
Publié: (2025)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
par: Wang, Haoxiang, et autres
Publié: (2024)

DSPA: Dynamic SAE Steering for Data-Efficient Preference Alignment
par: Wedgwood, James, et autres
Publié: (2026)

Can Brain Signals Reveal Inner Alignment with Human Languages?
par: Han, William, et autres
Publié: (2022)

Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization
par: Huang, Audrey, et autres
Publié: (2024)

Auto-ICL: In-Context Learning without Human Supervision
par: Yang, Jinghan, et autres
Publié: (2023)

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs
par: Sun, Hao, et autres
Publié: (2025)

Active Preference Learning for Large Language Models
par: Muldrew, William, et autres
Publié: (2024)

Evolutionary Contrastive Distillation for Language Model Alignment
par: Katz-Samuels, Julian, et autres
Publié: (2024)

Course-Correction: Safety Alignment Using Synthetic Preferences
par: Xu, Rongwu, et autres
Publié: (2024)

MixDPO: Modeling Preference Strength for Pluralistic Alignment
par: Imai, Saki, et autres
Publié: (2026)

Less is More: Improving LLM Alignment via Preference Data Selection
par: Deng, Xun, et autres
Publié: (2025)

Teaching Your Models to Understand Code via Focal Preference Alignment
par: Wu, Jie, et autres
Publié: (2025)

Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
par: Yang, Rui, et autres
Publié: (2024)

Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
par: Liu, Yinhong, et autres
Publié: (2024)

Would I Lie To You? Inference Time Alignment of Language Models using Direct Preference Heads
par: Hadji-Kyriacou, Avelina Asada, et autres
Publié: (2024)

Answer Matching Outperforms Multiple Choice for Language Model Evaluation
par: Chandak, Nikhil, et autres
Publié: (2025)

Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning
par: Rajaram, Sara, et autres
Publié: (2025)

Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences
par: Pattnaik, Pulkit, et autres
Publié: (2024)

AdapThink: Adaptive Thinking Preferences for Reasoning Language Model
par: Wan, Xu, et autres
Publié: (2025)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
par: Kim, Dongyoung, et autres
Publié: (2024)

Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
par: Liu, Chris Yuhao, et autres
Publié: (2025)

Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models
par: Wang, Fei, et autres
Publié: (2024)

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
par: Gong, Ruihao, et autres
Publié: (2024)

Self-Supervised Visual Preference Alignment
par: Zhu, Ke, et autres
Publié: (2024)

Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
par: D'Oosterlinck, Karel, et autres
Publié: (2024)