Cover Image

Shift is Good: Mismatched Data Mixing Improves Test Performance

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Medvedev, Marko, Lyu, Kaifeng, Li, Zhiyuan, Srebro, Nathan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Weak-to-Strong Generalization Even in Random Feature Networks, Provably
by: Medvedev, Marko, et al.
Published: (2025)

Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality
by: Medvedev, Marko, et al.
Published: (2024)

Positive Distribution Shift as a Framework for Understanding Tractable Learning
by: Medvedev, Marko, et al.
Published: (2026)

Recursive Models for Long-Horizon Reasoning
by: Yang, Chenxiao, et al.
Published: (2026)

Quantifying Overfitting along the Regularization Path for Two-Part-Code MDL in Supervised Classification
by: Zhu, Xiaohan, et al.
Published: (2025)

PENCIL: Long Thoughts with Short Memory
by: Yang, Chenxiao, et al.
Published: (2025)

Data Mixing Can Induce Phase Transitions in Knowledge Acquisition
by: Gu, Xinran, et al.
Published: (2025)

Noisy Interpolation Learning with Shallow Univariate ReLU Networks
by: Joshi, Nirmit, et al.
Published: (2023)

Overfitting and Generalizing with (PAC) Bayesian Prediction in Noisy Binary Classification
by: Zhu, Xiaohan, et al.
Published: (2026)

The Implicit Bias of Gradient Descent on Separable Data
by: Soudry, Daniel, et al.
Published: (2017)

On the Complexity of Learning Sparse Functions with Statistical and Gradient Queries
by: Joshi, Nirmit, et al.
Published: (2024)

Tight Bounds on the Binomial CDF, and the Minimum of i.i.d Binomials, in terms of KL-Divergence
by: Zhu, Xiaohan, et al.
Published: (2025)

The Price of Implicit Bias in Adversarially Robust Generalization
by: Tsilivis, Nikolaos, et al.
Published: (2024)

Research Program: Theory of Learning in Dynamical Systems
by: Hazan, Elad, et al.
Published: (2025)

Online Bandits with (Biased) Offline Data: Adaptive Learning under Distribution Mismatch
by: Cheung, Wang Chi, et al.
Published: (2024)

The Marginal Value of Momentum for Small Learning Rate SGD
by: Wang, Runzhe, et al.
Published: (2023)

An Agnostic View on the Cost of Overfitting in (Kernel) Ridge Regression
by: Zhou, Lijia, et al.
Published: (2023)

On the Hardness of Learning Regular Expressions
by: Attias, Idan, et al.
Published: (2025)

Learning single-index models via harmonic decomposition
by: Joshi, Nirmit, et al.
Published: (2025)

Adam Reduces a Unique Form of Sharpness: Theoretical Insights Near the Minimizer Manifold
by: Li, Xinghan, et al.
Published: (2025)

A Theory of Learning with Autoregressive Chain of Thought
by: Joshi, Nirmit, et al.
Published: (2025)

Temperature is All You Need for Generalization in Langevin Dynamics and other Markov Processes
by: Harel, Itamar, et al.
Published: (2025)

Depth Separation in Norm-Bounded Infinite-Width Neural Networks
by: Parkinson, Suzanna, et al.
Published: (2024)

Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks
by: Li, Binghui, et al.
Published: (2024)

The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback
by: Lambert, Nathan, et al.
Published: (2023)

Mixed-feature Logistic Regression Robust to Distribution Shifts
by: Sun, Qingshi, et al.
Published: (2025)

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking
by: Lyu, Kaifeng, et al.
Published: (2023)

Cross-Domain Policy Adaptation by Capturing Representation Mismatch
by: Lyu, Jiafei, et al.
Published: (2024)

How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers
by: Buzaglo, Gon, et al.
Published: (2024)

Provable Tempered Overfitting of Minimal Nets and Typical Nets
by: Harel, Itamar, et al.
Published: (2024)

RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval
by: Wen, Kaiyue, et al.
Published: (2024)

Learning to Think from Multiple Thinkers
by: Joshi, Nirmit, et al.
Published: (2026)

Score Design for Multi-Criteria Incentivization
by: Kabra, Anmol, et al.
Published: (2024)

Learning to Answer from Correct Demonstrations
by: Joshi, Nirmit, et al.
Published: (2025)

Dataset Distillers Are Good Label Denoisers In the Wild
by: Cheng, Lechao, et al.
Published: (2024)

On the SDEs and Scaling Rules for Adaptive Gradient Algorithms
by: Malladi, Sadhika, et al.
Published: (2022)

From Continual Learning to SGD and Back: Better Rates for Continual Linear Models
by: Evron, Itay, et al.
Published: (2025)

On the Empirical Power of Goodness-of-Fit Tests in Watermark Detection
by: He, Weiqing, et al.
Published: (2025)

Understanding Uncertainty-based Active Learning Under Model Mismatch
by: Rahmati, Amir Hossein, et al.
Published: (2024)

Metalearning with Very Few Samples Per Task
by: Aliakbarpour, Maryam, et al.
Published: (2023)