Copertina

Pretrain-Test Task Alignment Governs Generalization in In-Context Learning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Letey, Mary I., Zavatone-Veth, Jacob A., Lu, Yue M., Pehlevan, Cengiz
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Asymptotic theory of in-context learning by linear attention
di: Lu, Yue M., et al.
Pubblicazione: (2024)

Nadaraya-Watson kernel smoothing as a random energy model
di: Zavatone-Veth, Jacob A., et al.
Pubblicazione: (2024)

Risk and cross validation in ridge regression with correlated samples
di: Atanasov, Alexander, et al.
Pubblicazione: (2024)

Scaling and renormalization in high-dimensional regression
di: Atanasov, Alexander, et al.
Pubblicazione: (2024)

Spectral regularization for adversarially-robust representation learning
di: Yang, Sheng, et al.
Pubblicazione: (2024)

Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time
di: Bordelon, Blake, et al.
Pubblicazione: (2025)

A Random Matrix Theory Perspective on the Consistency of Diffusion Models
di: Wang, Binxu, et al.
Pubblicazione: (2026)

Dynamically Learning to Integrate in Recurrent Neural Networks
di: Bordelon, Blake, et al.
Pubblicazione: (2025)

How does training shape the Riemannian geometry of neural network representations?
di: Zavatone-Veth, Jacob A., et al.
Pubblicazione: (2023)

A note on the dynamics of extended-context disordered kinetic spin models
di: Zavatone-Veth, Jacob A., et al.
Pubblicazione: (2025)

Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
di: Atanasov, Alexander, et al.
Pubblicazione: (2025)

Summary statistics of learning link changing neural representations to behavior
di: Zavatone-Veth, Jacob A., et al.
Pubblicazione: (2025)

Stimulus symmetries can confound representational similarity analyses
di: Pashakhanloo, Farhad, et al.
Pubblicazione: (2026)

MLPs Learn In-Context on Regression and Classification Tasks
di: Tong, William L., et al.
Pubblicazione: (2024)

A solvable model of learning generative diffusion: theory and insights
di: Cui, Hugo, et al.
Pubblicazione: (2025)

Disordered Dynamics in High Dimensions: Connections to Random Matrices and Machine Learning
di: Bordelon, Blake, et al.
Pubblicazione: (2026)

Boule or Baguette? A Study on Task Topology, Length Generalization, and the Benefit of Reasoning Traces
di: Tong, William L., et al.
Pubblicazione: (2026)

Learning Curves for Noisy Heterogeneous Feature-Subsampled Ridge Ensembles
di: Ruben, Benjamin S., et al.
Pubblicazione: (2023)

Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scaling
di: Halder, Indranil, et al.
Pubblicazione: (2025)

Learning richness modulates equality reasoning in neural networks
di: Tong, William L., et al.
Pubblicazione: (2025)

Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging
di: Meterez, Alexandru, et al.
Pubblicazione: (2026)

Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
di: Bordelon, Blake, et al.
Pubblicazione: (2025)

Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
di: Zhao, Rosie, et al.
Pubblicazione: (2025)

Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism
di: Erdogan, Mete, et al.
Pubblicazione: (2025)

Transfer Learning in Infinite Width Feature Learning Networks
di: Lauditi, Clarissa, et al.
Pubblicazione: (2025)

How Feature Learning Can Improve Neural Scaling Laws
di: Bordelon, Blake, et al.
Pubblicazione: (2024)

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
di: Lauditi, Clarissa, et al.
Pubblicazione: (2026)

Convex Relaxation for Solving Large-Margin Classifiers in Hyperbolic Space
di: Yang, Sheng, et al.
Pubblicazione: (2024)

Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover
di: Halder, Indranil, et al.
Pubblicazione: (2026)

The Optimization Landscape of SGD Across the Feature Learning Strength
di: Atanasov, Alexander, et al.
Pubblicazione: (2024)

Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
di: Uzun, Mustafa, et al.
Pubblicazione: (2026)

Universal One-third Time Scaling in Learning Peaked Distributions
di: Liu, Yizhou, et al.
Pubblicazione: (2026)

An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models
di: Wang, Binxu, et al.
Pubblicazione: (2025)

Adaptive kernel predictors from feature-learning infinite limits of neural networks
di: Lauditi, Clarissa, et al.
Pubblicazione: (2025)

A Dynamical Model of Neural Scaling Laws
di: Bordelon, Blake, et al.
Pubblicazione: (2024)

Hyperparameter Transfer with Mixture-of-Expert Layers
di: Jiang, Tianze, et al.
Pubblicazione: (2026)

Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry
di: Bozkurt, Bariscan, et al.
Pubblicazione: (2023)

Infinite Limits of Multi-head Transformer Dynamics
di: Bordelon, Blake, et al.
Pubblicazione: (2024)

Pixel-Based Similarities as an Alternative to Neural Data for Improving Convolutional Neural Network Adversarial Robustness
di: Attias, Elie, et al.
Pubblicazione: (2024)

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
di: Wu, Jingfeng, et al.
Pubblicazione: (2023)