Image de couverture de livre

The Viscosity of Logic: Phase Transitions and Hysteresis in DPO Alignment

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Pollanen, Marco
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Latent-Hysteresis Graph ODEs: Modeling Coupled Topology-Feature Evolution via Continuous Phase Transitions
par: Hou, Qinhan, et autres
Publié: (2026)

ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
par: Lin, Xiaoqiang, et autres
Publié: (2025)

Conditional Equivalence of DPO and RLHF: Implicit Assumption, Failure Modes, and Provable Alignment
par: Yang, Zhiqin, et autres
Publié: (2026)

MixDPO: Modeling Preference Strength for Pluralistic Alignment
par: Imai, Saki, et autres
Publié: (2026)

A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO
par: Zhou, Xingyu, et autres
Publié: (2025)

Provably Mitigating Corruption, Overoptimization, and Verbosity Simultaneously in Offline and Online RLHF/DPO Alignment
par: Chen, Ziyi, et autres
Publié: (2025)

Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences
par: Pattnaik, Pulkit, et autres
Publié: (2024)

What Matters in Data for DPO?
par: Pan, Yu, et autres
Publié: (2025)

Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
par: Li, Jiping, et autres
Publié: (2025)

daDPO: Distribution-Aware DPO for Distilling Conversational Abilities
par: Zhang, Zhengze, et autres
Publié: (2025)

VideoDPO: Omni-Preference Alignment for Video Diffusion Generation
par: Liu, Runtao, et autres
Publié: (2024)

SP^2DPO: An LLM-assisted Semantic Per-Pair DPO Generalization
par: He, Chaoyue, et autres
Publié: (2026)

$β$-DPO: Direct Preference Optimization with Dynamic $β$
par: Wu, Junkang, et autres
Publié: (2024)

C2-DPO: Constrained Controlled Direct Preference Optimization
par: Asadi, Kavosh, et autres
Publié: (2025)

Lying Is Just a Phase: The Hidden Alignment Transition in Language Model Scaling
par: Amin, Adil
Publié: (2026)

Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment
par: Li, Cheryl, et autres
Publié: (2025)

$ξ$-DPO: Direct Preference Optimization via Ratio Reward Margin
par: Fan, Zhengyuan, et autres
Publié: (2026)

Random Is Hard to Beat: Active Selection in online DPO with Modern LLMs
par: Oh, Giyeong, et autres
Publié: (2026)

Multitask Kernel-based Learning with Logic Constraints
par: Diligenti, Michelangelo, et autres
Publié: (2024)

SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety
par: Kim, Geon-Hyeong, et autres
Publié: (2025)

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization
par: Rho, Hyung Gyu
Publié: (2025)

GFRIEND: Generative Few-shot Reward Inference through EfficieNt DPO
par: Zhao, Yiyang, et autres
Publié: (2025)

GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Models
par: Holderrieth, Peter, et autres
Publié: (2025)

Catapult Dynamics and Phase Transitions in Quadratic Nets
par: Meltzer, David, et autres
Publié: (2023)

DPO Meets PPO: Reinforced Token Optimization for RLHF
par: Zhong, Han, et autres
Publié: (2024)

How Explanations Leak the Decision Logic: Stealing Graph Neural Networks via Explanation Alignment
par: Ma, Bin, et autres
Publié: (2025)

EnerBridge-DPO: Energy-Guided Protein Inverse Folding with Markov Bridges and Direct Preference Optimization
par: Rong, Dingyi, et autres
Publié: (2025)

Aligning Compound AI Systems via System-level DPO
par: Wang, Xiangwen, et autres
Publié: (2025)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
par: Wu, Junkang, et autres
Publié: (2024)

Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
par: Pal, Arka, et autres
Publié: (2024)

Fairness is Not Flat: Geometric Phase Transitions Against Shortcut Learning
par: Rodriguez-Alvarez, Nicolas, et autres
Publié: (2026)

Towards Generalisable Imitation Learning Through Conditioned Transition Estimation and Online Behaviour Alignment
par: Gavenski, Nathan, et autres
Publié: (2026)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
par: Qi, Xuan, et autres
Publié: (2025)

Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing
par: Qi, Biqing, et autres
Publié: (2024)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
par: Wang, Bo, et autres
Publié: (2025)

Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering
par: Mohamed, Anas, et autres
Publié: (2025)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
par: Gupta, Taneesh, et autres
Publié: (2024)

Decomposing Behavioral Phase Transitions in LLMs: Order Parameters for Emergent Misalignment
par: Arnold, Julian, et autres
Publié: (2025)

eXpLogic: Explaining Logic Types and Patterns in DiffLogic Networks
par: Wormald, Stephen, et autres
Publié: (2025)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
par: Lai, Xin, et autres
Publié: (2024)