Cover Image

Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jones-McCormick, Taj, Jagannath, Aukosh, Sen, Subhabrata
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
by: Jagannath, Aukosh, et al.
Published: (2025)

Detecting Metastable Basins in High Dimensions via Marginal Trajectory Distribution Discrimination
by: Jones-McCormick, Taj
Published: (2026)

Universality of high-dimensional scaling limits of stochastic gradient descent
by: Gheissari, Reza, et al.
Published: (2025)

Optimality of Message-Passing Architectures for Sparse Graphs
by: Baranwal, Aseem, et al.
Published: (2023)

Multi-layer Cross-attention is Provably Optimal for Multi-modal In-context Learning
by: Barnfield, Nicholas, et al.
Published: (2026)

Spectral alignment of stochastic gradient descent for high-dimensional classification tasks
by: Arous, Gerard Ben, et al.
Published: (2023)

Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions
by: Arous, Gerard Ben, et al.
Published: (2025)

Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis
by: Li, Yufan, et al.
Published: (2024)

Unique Rashomon Sets for Robust Active Learning
by: Nguyen, Simon, et al.
Published: (2025)

Differentially private multivariate medians
by: Ramsay, Kelly, et al.
Published: (2022)

Adaptive Active Learning for Regression via Reinforcement Learning
by: Nguyen, Simon D., et al.
Published: (2026)

On Provable Benefits of Muon in Federated Learning
by: Zhang, Xinwen, et al.
Published: (2025)

A Unified Framework for Inference with General Missingness Patterns and Machine Learning Imputation
by: Chen, Xingran, et al.
Published: (2025)

Do Pre-trained Models Benefit Equally in Continual Learning?
by: Lee, Kuan-Ying, et al.
Published: (2022)

Provable Benefits of In-Tool Learning for Large Language Models
by: Houliston, Sam, et al.
Published: (2025)

PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning
by: Ying, Chengyang, et al.
Published: (2024)

Spatially Robust Inference with Predicted and Missing at Random Labels
by: Salerno, Stephen, et al.
Published: (2026)

Transfer Learning with Pre-trained Conditional Generative Models
by: Yamaguchi, Shin'ya, et al.
Published: (2022)

Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum
by: Rajaraman, Nived, et al.
Published: (2026)

Bayes optimal learning in high-dimensional linear regression with network side information
by: Nandy, Sagnik, et al.
Published: (2023)

Distributed Learning and Inference Systems: A Networking Perspective
by: Moussa, Hesham G., et al.
Published: (2025)

Endowing Pre-trained Graph Models with Provable Fairness
by: Zhang, Zhongjian, et al.
Published: (2024)

REALITrees: Rashomon Ensemble Active Learning for Interpretable Trees
by: Nguyen, Simon D., et al.
Published: (2026)

Time-Series Classification in Smart Manufacturing Systems: An Experimental Evaluation of State-of-the-Art Machine Learning Algorithms
by: Farahani, Mojtaba A., et al.
Published: (2023)

Provable Benefit of Cutout and CutMix for Feature Learning
by: Oh, Junsoo, et al.
Published: (2024)

High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models
by: Liang, Tengyuan, et al.
Published: (2022)

SOPHON: Non-Fine-Tunable Learning to Restrain Task Transferability For Pre-trained Models
by: Deng, Jiangyi, et al.
Published: (2024)

Spectral goodness-of-fit tests for complete and partial network data
by: Lubold, Shane, et al.
Published: (2021)

Provable Benefits of Sinusoidal Activation for Modular Addition
by: Huang, Tianlong, et al.
Published: (2025)

Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learning
by: Cheng, Ziheng, et al.
Published: (2025)

To Stay or Not to Stay in the Pre-train Basin: Insights on Ensembling in Transfer Learning
by: Sadrtdinov, Ildus, et al.
Published: (2023)

Unsupervised Learning of Efficient Exploration: Pre-training Adaptive Policies via Self-Imposed Goals
by: Pappalardo, Octavio
Published: (2026)

AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models
by: He, Run, et al.
Published: (2024)

Physics-Informed Neural Networks for Electrical Circuit Analysis: Applications in Dielectric Material Modeling
by: Taj, Reyhaneh
Published: (2024)

Momentum Benefits Non-IID Federated Learning Simply and Provably
by: Cheng, Ziheng, et al.
Published: (2023)

Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator
by: Luo, Beier, et al.
Published: (2025)

Decoupling Exploration and Exploitation for Unsupervised Pre-training with Successor Features
by: Kim, JaeYoon, et al.
Published: (2024)

Cross-Domain Pre-training with Language Models for Transferable Time Series Representations
by: Cheng, Mingyue, et al.
Published: (2024)

Do We Really Even Need Data?
by: Hoffman, Kentaro, et al.
Published: (2024)

Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training
by: Bu, Dake, et al.
Published: (2025)