Cover Image

OPTune: Efficient Online Preference Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Lichang, Chen, Jiuhai, Liu, Chenxi, Kirchenbauer, John, Soselia, Davit, Zhu, Chen, Goldstein, Tom, Zhou, Tianyi, Huang, Heng
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ODIN: Disentangled Reward Mitigates Hacking in RLHF
by: Chen, Lichang, et al.
Published: (2024)

Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements
by: Li, Ming, et al.
Published: (2024)

Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
by: Li, Ming, et al.
Published: (2024)

GenQA: Generating Millions of Instructions from a Handful of Prompts
by: Chen, Jiuhai, et al.
Published: (2024)

From Lists to Emojis: How Format Bias Affects Model Alignment
by: Zhang, Xuanchang, et al.
Published: (2024)

FictionalQA: A Dataset for Studying Memorization and Knowledge Acquisition
by: Kirchenbauer, John, et al.
Published: (2025)

Modality-Balancing Preference Optimization of Large Multimodal Models by Adversarial Negative Mining
by: Liu, Chenxi, et al.
Published: (2025)

Multi-Token Prediction via Self-Distillation
by: Kirchenbauer, John, et al.
Published: (2026)

A Watermark for Large Language Models
by: Kirchenbauer, John, et al.
Published: (2023)

Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement
by: Wang, Xiyao, et al.
Published: (2024)

LMD3: Language Model Data Density Dependence
by: Kirchenbauer, John, et al.
Published: (2024)

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
by: Geiping, Jonas, et al.
Published: (2025)

From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning
by: Li, Ming, et al.
Published: (2023)

Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection
by: Chen, Ruibo, et al.
Published: (2024)

Antidistillation Fingerprinting
by: Xu, Yixuan Even, et al.
Published: (2026)

ComPO: Preference Alignment via Comparison Oracles
by: Chen, Peter, et al.
Published: (2025)

GATES: Self-Distillation under Privileged Context with Consensus Gating
by: Stein, Alex, et al.
Published: (2026)

Multi-Objective Linguistic Control of Large Language Models
by: Nguyen, Dang, et al.
Published: (2024)

ATLaS: Agent Tuning via Learning Critical Steps
by: Chen, Zhixun, et al.
Published: (2025)

On the Reliability of Watermarks for Large Language Models
by: Kirchenbauer, John, et al.
Published: (2023)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
by: Fernando, Heshan, et al.
Published: (2024)

A Unified Understanding of Offline Data Selection and Online Self-refining Generation for Post-training LLMs
by: Xiao, Quan, et al.
Published: (2025)

ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning
by: Melikidze, Davit, et al.
Published: (2026)

LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
by: Liu, Yezi, et al.
Published: (2025)

Automated Data Curation for Robust Language Model Fine-Tuning
by: Chen, Jiuhai, et al.
Published: (2024)

Homonym Sense Disambiguation in the Georgian Language
by: Melikidze, Davit, et al.
Published: (2024)

Contrastive Instruction Tuning
by: Yan, Tianyi Lorena, et al.
Published: (2024)

The Crucial Role of Samplers in Online Direct Preference Optimization
by: Shi, Ruizhe, et al.
Published: (2024)

Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection
by: Yan, Jun, et al.
Published: (2023)

Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
by: McLeish, Sean, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning via Circular Convolution
by: Chen, Aochuan, et al.
Published: (2024)

LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning
by: Huang, Yining, et al.
Published: (2025)

Preference Optimization with Multi-Sample Comparisons
by: Wang, Chaoqi, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning with Discrete Fourier Transform
by: Gao, Ziqi, et al.
Published: (2024)

Generalizing Reward Modeling for Out-of-Distribution Preference Learning
by: Jia, Chen
Published: (2024)

Memory-Efficient LLM Training with Online Subspace Descent
by: Liang, Kaizhao, et al.
Published: (2024)

CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad
by: Chen, Yongqiang, et al.
Published: (2026)

ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness
by: Liang, Yijun, et al.
Published: (2025)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
by: Qiu, Wenjie, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain
by: Gema, Aryo Pradipta, et al.
Published: (2023)