Cover Image

A Principled Loss Function for Direct Language Model Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Tan, Yuandong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokking
by: Tian, Yuandong
Published: (2025)

Param$Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost
by: Cao, Sheng, et al.
Published: (2025)

Sail into the Headwind: Alignment via Robust Rewards and Dynamic Labels against Reward Hacking
by: Rashidinejad, Paria, et al.
Published: (2024)

From Coefficients to Directions: Rethinking Model Merging with Directional Alignment
by: Chen, Zhikang, et al.
Published: (2025)

Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learning
by: Ding, Zihan, et al.
Published: (2024)

SHARP: Accelerating Language Model Inference by SHaring Adjacent layers with Recovery Parameters
by: Wang, Yiping, et al.
Published: (2025)

Direct Alignment with Heterogeneous Preferences
by: Shirali, Ali, et al.
Published: (2025)

Inference-Time Alignment of Diffusion Models with Direct Noise Optimization
by: Tang, Zhiwei, et al.
Published: (2024)

Align-Pro: A Principled Approach to Prompt Optimization for LLM Alignment
by: Trivedi, Prashant, et al.
Published: (2025)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

A Note on Loss Functions and Error Compounding in Model-based Reinforcement Learning
by: Jiang, Nan
Published: (2024)

Doubly Robust Alignment for Large Language Models
by: Xu, Erhan, et al.
Published: (2025)

A Probabilistic Perspective on Unlearning and Alignment for Large Language Models
by: Scholten, Yan, et al.
Published: (2024)

Meta-Learning Adaptive Loss Functions
by: Raymond, Christian, et al.
Published: (2023)

Towards General-Purpose Model-Free Reinforcement Learning
by: Fujimoto, Scott, et al.
Published: (2025)

Geometry Preserving Loss Functions Promote Improved Adaptation of Blackbox Generative Model
by: Mitra, Sinjini, et al.
Published: (2026)

Evidence-based Distributional Alignment for Large Language Models
by: Pham, Viet-Thanh, et al.
Published: (2026)

A Note on Knowledge Distillation Loss Function for Object Classification
by: Chen, Defang
Published: (2021)

Visualizing Loss Functions as Topological Landscape Profiles
by: Geniesse, Caleb, et al.
Published: (2024)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
by: Zhang, Shenao, et al.
Published: (2024)

SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment
by: Zhu, Wenqiao, et al.
Published: (2025)

Would I Lie To You? Inference Time Alignment of Language Models using Direct Preference Heads
by: Hadji-Kyriacou, Avelina Asada, et al.
Published: (2024)

RSPO: Regularized Self-Play Alignment of Large Language Models
by: Tang, Xiaohang, et al.
Published: (2025)

Offline Regularised Reinforcement Learning for Large Language Models Alignment
by: Richemond, Pierre Harvey, et al.
Published: (2024)

SOM Directions are Better than One: Multi-Directional Refusal Suppression in Language Models
by: Piras, Giorgio, et al.
Published: (2025)

Context-Adaptive Multi-Prompt Embedding with Large Language Models for Vision-Language Alignment
by: Kim, Dahun, et al.
Published: (2025)

AI Alignment with Changing and Influenceable Reward Functions
by: Carroll, Micah, et al.
Published: (2024)

Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms
by: Rafailov, Rafael, et al.
Published: (2024)

ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
by: Lin, Xiaoqiang, et al.
Published: (2025)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
by: Xu, Zaiyan, et al.
Published: (2025)

Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining
by: Sow, Daouda, et al.
Published: (2025)

Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?
by: Gu, Zhuojun, et al.
Published: (2025)

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment
by: Wang, Chaoqi, et al.
Published: (2025)

Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Models
by: Guo, Hanze, et al.
Published: (2025)

Self-Exploring Language Models: Active Preference Elicitation for Online Alignment
by: Zhang, Shenao, et al.
Published: (2024)

Human Alignment of Large Language Models through Online Preference Optimisation
by: Calandriello, Daniele, et al.
Published: (2024)

Semantic-aware Wasserstein Policy Regularization for Large Language Model Alignment
by: Na, Byeonghu, et al.
Published: (2026)

In Search for Architectures and Loss Functions in Multi-Objective Reinforcement Learning
by: Terekhov, Mikhail, et al.
Published: (2024)

Beyond the Mean: Distribution-Aware Loss Functions for Bimodal Regression
by: Mohammadi-Seif, Abolfazl, et al.
Published: (2026)

The Blessing and Curse of Dimensionality in Safety Alignment
by: Teo, Rachel S. Y., et al.
Published: (2025)