Cover Image

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Guo, Yiju, Cui, Ganqu, Yuan, Lifan, Ding, Ning, Sun, Zexu, Sun, Bowen, Chen, Huimin, Xie, Ruobing, Zhou, Jie, Lin, Yankai, Liu, Zhiyuan, Sun, Maosong
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Systems and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Advancing LLM Reasoning Generalists with Preference Trees
by: Yuan, Lifan, et al.
Published: (2024)

Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruning
by: Guo, Yiju, et al.
Published: (2025)

UltraFeedback: Boosting Language Models with Scaled AI Feedback
by: Cui, Ganqu, et al.
Published: (2023)

Less Noise, More Voice: Reinforcement Learning for Reasoning via Instruction Purification
by: Guo, Yiju, et al.
Published: (2026)

Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models
by: Ding, Ning, et al.
Published: (2024)

From $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old Ones
by: Yuan, Lifan, et al.
Published: (2025)

Representation Learning for Natural Language Processing
by: Liu, Zhiyuan, et al.
Published: (2020)

Meta-Aligner: Bidirectional Preference-Policy Optimization for Multi-Objective LLMs Alignment
by: Xu, Wenzhe, et al.
Published: (2026)

The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning
by: He, Bingxiang, et al.
Published: (2024)

AIR: A Systematic Analysis of Annotations, Instructions, and Response Pairs in Preference Dataset
by: He, Bingxiang, et al.
Published: (2025)

RLPR: Extrapolating RLVR to General Domains without Verifiers
by: Yu, Tianyu, et al.
Published: (2025)

Free Process Rewards without Process Labels
by: Yuan, Lifan, et al.
Published: (2024)

Exploring the Benefit of Activation Sparsity in Pre-training
by: Zhang, Zhengyan, et al.
Published: (2024)

The Overthinker's DIET: Cutting Token Calories with DIfficulty-AwarE Training
by: Chen, Weize, et al.
Published: (2025)

Noise Contrastive Alignment of Language Models with Explicit Rewards
by: Chen, Huayu, et al.
Published: (2024)

RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
by: Yu, Tianyu, et al.
Published: (2023)

LaSeR: Reinforcement Learning with Last-Token Self-Rewarding
by: Yang, Wenkai, et al.
Published: (2025)

Empowering Private Tutoring by Chaining Large Language Models
by: Chen, Yulin, et al.
Published: (2023)

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
by: Xiao, Chaojun, et al.
Published: (2023)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
by: Zhou, Zhanhui, et al.
Published: (2023)

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System
by: Chen, Weize, et al.
Published: (2024)

M$^3$TN: Multi-gate Mixture-of-Experts based Multi-valued Treatment Network for Uplift Modeling
by: Sun, Zexu, et al.
Published: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization
by: Fu, Yuhan, et al.
Published: (2024)

Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication
by: Chen, Weize, et al.
Published: (2024)

Quality-Diversity Optimization as Multi-Objective Optimization
by: Lin, Xi, et al.
Published: (2026)

H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
by: Gao, Cheng, et al.
Published: (2025)

Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs
by: Gao, Cheng, et al.
Published: (2024)

TTRL: Test-Time Reinforcement Learning
by: Zuo, Yuxin, et al.
Published: (2025)

Preference Orchestrator: Prompt-Aware Multi-Objective Alignment for Large Language Models
by: Liu, Biao, et al.
Published: (2025)

Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention
by: Lv, Xingtai, et al.
Published: (2024)

Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models
by: Agnihotri, Akhil, et al.
Published: (2025)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs
by: Gao, Rui, et al.
Published: (2026)

Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents
by: Qian, Cheng, et al.
Published: (2024)

Lean Refactor: Multi-Objective Controllable Proof Optimization via Agentic Strategy Search
by: Lu, Jialin, et al.
Published: (2026)

Listwise Direct Preference Optimization with Multi-Dimensional Preference Mixing
by: Sun, Yuhui, et al.
Published: (2025)

Self-Improvement Towards Pareto Optimality: Mitigating Preference Conflicts in Multi-Objective Alignment
by: Li, Moxin, et al.
Published: (2025)

Robust Multi-Objective Preference Alignment with Online DPO
by: Gupta, Raghav, et al.
Published: (2025)

INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair
by: Wang, Hanbin, et al.
Published: (2023)