Cover Image

Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Chia-Hsuan, Zhou, Mingyang, Ni, Renkun, Cheng, Zelei, Dai, Sihui, Chakraborty, Supriyo, Zhang, Shixiong, Sahu, Sambit, Campbell, William
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for Efficient Financial LLM Adaptation
by: Kapusuzoglu, Berkcan, et al.
Published: (2025)

Critique-Guided Distillation for Robust Reasoning via Refinement
by: Kapusuzoglu, Berkcan, et al.
Published: (2025)

Your Model Diversity, Not Method, Determines Reasoning Strategy
by: Choraria, Moulik, et al.
Published: (2026)

Optimizing Reasoning Efficiency through Prompt Difficulty Prediction
by: Zhao, Bo, et al.
Published: (2025)

Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning
by: Mircea, Andrei, et al.
Published: (2025)

Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
by: Panda, Ashwinee, et al.
Published: (2025)

What Do Latent Action Models Actually Learn?
by: Zhang, Chuheng, et al.
Published: (2025)

Interview of the famous female impersonator of Indian Theatre: Chapal Bhaduri
by: Supriyo Chakraborty
Published: (2019)

LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Models
by: Veldanda, Akshaj Kumar, et al.
Published: (2024)

Leveraging Parameter Space Symmetries for Reasoning Skill Transfer in LLMs
by: Horoi, Stefan, et al.
Published: (2025)

What Do GNNs Actually Learn? Towards Understanding their Representations
by: Nikolentzos, Giannis, et al.
Published: (2023)

Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimization
by: Yu, Jiahao, et al.
Published: (2025)

Writing at the Graduate Level: What Tasks Do Professors Actually Require?
by: Cooper, Amy, et al.
Published: (2007)

What Does Deep Hedging Actually Learn? Delta Corrections, Regime Fragility, and Symbolic Distillation
by: Zernikov, Kirill
Published: (2026)

CoT-Guard: Small Models for Strong Monitoring
by: Diwan, Nirav, et al.
Published: (2026)

Prediction Bottlenecks Don't Discover Causal Structure (But Here's What They Actually Do)
by: Lade, Ankit Hemant, et al.
Published: (2026)

A Multiple‐Choice Exercise on Collocations: What Do Learners Actually Remember?
by: Alyssa Mengxue Li, et al.
Published: (2024)

What Work is AI Actually Doing? Uncovering the Drivers of Generative AI Adoption
by: Agarwal, Peeyush, et al.
Published: (2025)

DIAL-SUMMER: A Structured Evaluation Framework of Hierarchical Errors in Dialogue Summaries
by: Ramnath, Sahana, et al.
Published: (2026)

"What Did It Actually Do?": Understanding Risk Awareness and Traceability for Computer-Use Agents
by: Peng, Zifan, et al.
Published: (2026)

What Do Claim Verification Datasets Actually Test? A Reasoning Trace Analysis
by: Rao, Delip, et al.
Published: (2026)

On Understanding of the Dynamics of Model Capacity in Continual Learning
by: Chakraborty, Supriyo, et al.
Published: (2025)

What Voting Rules Actually Do: A Data-Driven Analysis of Multi-Winner Voting
by: Caiata, Joshua, et al.
Published: (2025)

What Does a Research Lead Actually Do? Reflections From the Emergency Department Frontline
by: Faye Jordan
Published: (2025)

Decomposed Direct Preference Optimization for Structure-Based Drug Design
by: Cheng, Xiwei, et al.
Published: (2024)

The Code Barrier: What LLMs Actually Understand?
by: Nikiema, Serge Lionel, et al.
Published: (2025)

What Do Patients Prefer? Therapy after Reverse Shoulder Arthroplasty
by: William Kardasis
Published: (2025)

PairAug: What Can Augmented Image-Text Pairs Do for Radiology?
by: Xie, Yutong, et al.
Published: (2024)

PairVDN - Pair-wise Decomposed Value Functions
by: Buzzard, Zak
Published: (2025)

Do CFLOBDDs Actually Make Use of Linear Structure?
by: Sistla, Meghana Aparna, et al.
Published: (2026)

A Survey on Explainable Deep Reinforcement Learning
by: Cheng, Zelei, et al.
Published: (2025)

Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection
by: Niu, Tianyi, et al.
Published: (2026)

What Is Preference Optimization Doing, and Why?
by: Wang, Yue, et al.
Published: (2025)

Borrowed Templates: Is What You're Reaching For Actually Yours?
by: Sriharan, Vaz
Published: (2026)

What Light Actually Carries: Frequency Registration and the Limits of Observation
by: Doumbouya, Lisa Michelle
Published: (2026)

What Serious Agentic AI Developers Actually Need to Know
by: Rosehill, Daniel, et al.
Published: (2026)

Barley Bread in the Bible: What Ancient Israelites Actually Ate
by: Rosehill, Daniel, et al.
Published: (2026)

What do Geometric Hallucination Detection Metrics Actually Measure?
by: Yeats, Eric, et al.
Published: (2026)

Lite-SAM Is Actually What You Need for Segment Everything
by: Fu, Jianhai, et al.
Published: (2024)

OrchestraLLM: Efficient Orchestration of Language Models for Dialogue State Tracking
by: Lee, Chia-Hsuan, et al.
Published: (2023)