Cover Image

From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Oh, Junsoo, Song, Jerry, Yun, Chulhee
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Provable Benefit of Cutout and CutMix for Feature Learning
by: Oh, Junsoo, et al.
Published: (2024)

DASH: Warm-Starting Neural Network Training in Stationary Settings without Loss of Plasticity
by: Shin, Baekrok, et al.
Published: (2024)

Weak-to-Strong Generalization Even in Random Feature Networks, Provably
by: Medvedev, Marko, et al.
Published: (2025)

Weak-to-Strong Generalization is Nearly Inevitable (in Linear Models)
by: Geng, Scott, et al.
Published: (2026)

Stochastic Extragradient with Flip-Flop Shuffling & Anchoring: Provable Improvements
by: Chae, Jiseok, et al.
Published: (2024)

Mamba Can Learn Low-Dimensional Targets In-Context via Test-Time Feature Learning
by: Oh, Junsoo, et al.
Published: (2025)

Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
by: Kim, Jihwan, et al.
Published: (2026)

Provable Weak-to-Strong Generalization via Benign Overfitting
by: Wu, David X., et al.
Published: (2024)

Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification
by: Jung, Hyunji, et al.
Published: (2025)

Understanding Sharpness Dynamics in NN Training with a Minimalist Example: The Effects of Dataset Difficulty, Depth, Stochasticity, and More
by: Yoo, Geonhui, et al.
Published: (2025)

Implicit Bias and Loss of Plasticity in Matrix Completion: Depth Promotes Low-Rankness
by: Shin, Baekrok, et al.
Published: (2026)

Does SGD really happen in tiny subspaces?
by: Song, Minhak, et al.
Published: (2024)

Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime
by: Baek, Beomhan, et al.
Published: (2025)

Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks
by: Nichani, Eshaan, et al.
Published: (2023)

Linear attention is (maybe) all you need (to understand transformer optimization)
by: Ahn, Kwangjun, et al.
Published: (2023)

Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation
by: Kim, Jinuk, et al.
Published: (2026)

Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
by: Cho, Hanseul, et al.
Published: (2024)

The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge
by: Awano, Ryoya, et al.
Published: (2026)

Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems
by: Kim, Yujun, et al.
Published: (2025)

The Cost of Robustness: Tighter Bounds on Parameter Complexity for Robust Memorization in ReLU Nets
by: Kim, Yujun, et al.
Published: (2025)

Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization
by: Moon, Chaewon, et al.
Published: (2026)

Fundamental Benefit of Alternating Updates in Minimax Optimization
by: Lee, Jaewook, et al.
Published: (2024)

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
by: Song, Minhak, et al.
Published: (2025)

Provable In-Context Learning of Nonlinear Regression with Transformers
by: Li, Hongbo, et al.
Published: (2025)

Minimalist Softmax Attention Provably Learns Constrained Boolean Functions
by: Hu, Jerry Yao-Chieh, et al.
Published: (2025)

Uniform Spectral Growth and Convergence of Muon in LoRA-Style Matrix Factorization
by: Kang, Changmin, et al.
Published: (2026)

Improved Scaling Laws via Weak-to-Strong Generalization in Random Feature Ridge Regression
by: Wu, Diyuan, et al.
Published: (2026)

On Weak-to-Strong Generalization and f-Divergence
by: Yao, Wei, et al.
Published: (2025)

Characteristic Learning for Provable One Step Generation
by: Ding, Zhao, et al.
Published: (2024)

Regularized Online RLHF with Generalized Bilinear Preferences
by: Lee, Junghyun, et al.
Published: (2026)

Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimension
by: Dong, Yijun, et al.
Published: (2025)

Reinforcement Learning with Function Approximation: From Linear to Nonlinear
by: Long, Jihao, et al.
Published: (2023)

Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty
by: Cho, Yeseul, et al.
Published: (2025)

The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibration
by: Yao, Wei, et al.
Published: (2025)

On the Blessing of Pre-training in Weak-to-Strong Generalization
by: Yao, Wei, et al.
Published: (2026)

Theoretical Analysis of Weak-to-Strong Generalization
by: Lang, Hunter, et al.
Published: (2024)

Quantifying the Gain in Weak-to-Strong Generalization
by: Charikar, Moses, et al.
Published: (2024)

On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspective
by: Moniri, Behrad, et al.
Published: (2025)

Trust Functions: Near-Lossless Weak-to-Strong Generalization by Learning When to Trust the Weak Teacher
by: Uzunoglu, Arda, et al.
Published: (2026)

On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)