Cover Image

Spurious Correlation Learning in Preference Optimization: Mechanisms, Consequences, and Mitigation via Tie Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Moya, Christian, Semendinger, Alex, Lin, Guang, Thornley, Elliott
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mitigating Spurious Correlations via Disagreement Probability
by: Han, Hyeonggeun, et al.
Published: (2024)

Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigation
by: Steinmann, David, et al.
Published: (2024)

Reward Learning From Preference With Ties
by: Liu, Jinsong, et al.
Published: (2024)

The Trap of Trajectory: Towards Understanding and Mitigating Spurious Correlations in Agentic Memory
by: Tang, Luoxi, et al.
Published: (2026)

Mitigating Spurious Correlations in NLI via LLM-Synthesized Counterfactuals and Dynamic Balanced Sampling
by: Jaimes, Christopher Román
Published: (2025)

Out of Spuriousity: Improving Robustness to Spurious Correlations without Group Annotations
by: Le, Phuong Quynh, et al.
Published: (2024)

Superclass-Guided Representation Disentanglement for Spurious Correlation Mitigation
by: Liu, Chenruo, et al.
Published: (2025)

Combating Spurious Correlations in Graph Interpretability via Self-Reflection
by: Cai, Kecheng, et al.
Published: (2026)

Severing Spurious Correlations with Data Pruning
by: Mulchandani, Varun, et al.
Published: (2025)

Revisiting Spurious Correlation in Domain Generalization
by: Qin, Bin, et al.
Published: (2024)

SCL-GNN: Towards Generalizable Graph Neural Networks via Spurious Correlation Learning
by: Zhang, Yuxiang, et al.
Published: (2026)

Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning
by: Chen, Yiwei, et al.
Published: (2025)

The Pragmatic Frames of Spurious Correlations in Machine Learning: Interpreting How and Why They Matter
by: Bell, Samuel J., et al.
Published: (2024)

Spurious Rewards: Rethinking Training Signals in RLVR
by: Shao, Rulin, et al.
Published: (2025)

Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness
by: Park, Subeen, et al.
Published: (2025)

Mitigating Preference Hacking in Policy Optimization with Pessimism
by: Gupta, Dhawal, et al.
Published: (2025)

Accelerating Approximate Thompson Sampling with Underdamped Langevin Monte Carlo
by: Zheng, Haoyang, et al.
Published: (2024)

The Shutdown Problem: An AI Engineering Puzzle for Decision Theorists
by: Thornley, Elliott
Published: (2024)

Shutdownable Agents through POST-Agency
by: Thornley, Elliott
Published: (2025)

Mitigating Mismatch within Reference-based Preference Optimization
by: Yuan, Suqin, et al.
Published: (2026)

Efficient Exploration for Iterative Nash Preference Optimization
by: Nan, Tianlong, et al.
Published: (2026)

PB$^2$: Preference Space Exploration via Population-Based Methods in Preference-Based Reinforcement Learning
by: Driss, Brahim, et al.
Published: (2025)

Not Only the Last-Layer Features for Spurious Correlations: All Layer Deep Feature Reweighting
by: Hameed, Humza Wajid, et al.
Published: (2024)

Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning
by: Yu, Yongcan, et al.
Published: (2026)

Identifying Spurious Correlations using Counterfactual Alignment
by: Cohen, Joseph Paul, et al.
Published: (2023)

Large Learning Rates Simultaneously Achieve Robustness to Spurious Correlations and Compressibility
by: Barsbey, Melih, et al.
Published: (2025)

Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning
by: Giovanelli, Joseph, et al.
Published: (2023)

Removing Spurious Correlation from Neural Network Interpretations
by: Fotouhi, Milad, et al.
Published: (2024)

Meta-Learning Objectives for Preference Optimization
by: Alfano, Carlo, et al.
Published: (2024)

Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation
by: Ghaznavi, Mahdi, et al.
Published: (2024)

A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima
by: Tang, Yiming, et al.
Published: (2025)

Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning
by: Zhang, Tianle, et al.
Published: (2024)

MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning
by: Lin, Yunze
Published: (2025)

CATS: Mitigating Correlation Shift for Multivariate Time Series Classification
by: Lin, Xiao, et al.
Published: (2025)

When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs
by: Wang, Shaowen, et al.
Published: (2025)

TSO: Self-Training with Scaled Preference Optimization
by: Chen, Kaihui, et al.
Published: (2024)

Debiasing Online Preference Learning via Preference Feature Preservation
by: Kim, Dongyoung, et al.
Published: (2025)

Fine-Tuning Diffusion Models for Molecular Generation via Reinforcement Learning and Fast Sampling
by: Lin, Guang, et al.
Published: (2026)

Shortcut to Nowhere: Demystifying Deep Spurious Regression
by: Xu, Guanrong, et al.
Published: (2026)

Bridging Explainability and Embeddings: BEE Aware of Spuriousness
by: Păduraru, Cristian Daniel, et al.
Published: (2024)