Cover Image

Toward Understanding In-context vs. In-weight Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chan, Bryan, Chen, Xinyi, György, András, Schuurmans, Dale
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning Continually by Spectral Regularization
by: Lewandowski, Alex, et al.
Published: (2024)

Spectral Ghost in Representation Learning: from Component Analysis to Self-Supervised Learning
by: Dai, Bo, et al.
Published: (2026)

Plastic Learning with Deep Fourier Features
by: Lewandowski, Alex, et al.
Published: (2024)

Learning to Reason Efficiently with Discounted Reinforcement Learning
by: Ayoub, Alex, et al.
Published: (2025)

Rectifying Regression in Reinforcement Learning
by: Ayoub, Alex, et al.
Published: (2025)

What Can Grokking Teach Us About Learning Under Nonstationarity?
by: Lyle, Clare, et al.
Published: (2025)

Beyond Statistical Learning: Exact Learning Is Essential for General Intelligence
by: György, András, et al.
Published: (2025)

Spectral Representation-based Reinforcement Learning
by: Gao, Chenxiao, et al.
Published: (2025)

Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning
by: Zhang, Hongming, et al.
Published: (2023)

Directions of Curvature as an Explanation for Loss of Plasticity
by: Lewandowski, Alex, et al.
Published: (2023)

Prior-Dependent Allocations for Bayesian Fixed-Budget Best-Arm Identification in Structured Bandits
by: Nguyen, Nicolas, et al.
Published: (2024)

Towards Understanding How Transformers Learn In-context Through a Representation Learning Lens
by: Ren, Ruifeng, et al.
Published: (2023)

Reinforcement Teaching
by: Muslimani, Calarina, et al.
Published: (2022)

Towards Understanding the Relationship between In-context Learning and Compositional Generalization
by: Han, Sungjun, et al.
Published: (2024)

Soft Preference Optimization: Aligning Language Models to Expert Distributions
by: Sharifnassab, Arsalan, et al.
Published: (2024)

Beyond Expectations: Learning with Stochastic Dominance Made Practical
by: Cen, Shicong, et al.
Published: (2024)

Stochastic Gradient Succeeds for Bandits
by: Mei, Jincheng, et al.
Published: (2024)

Ordering-based Conditions for Global Convergence of Policy Gradient Methods
by: Mei, Jincheng, et al.
Published: (2025)

DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning
by: Zhou, Zijian, et al.
Published: (2024)

Partition Tree Weighting for Non-Stationary Stochastic Bandits
by: Veness, Joel, et al.
Published: (2025)

Small steps no more: Global convergence of stochastic gradient bandits for arbitrary learning rates
by: Mei, Jincheng, et al.
Published: (2025)

To Believe or Not to Believe Your LLM
by: Yadkori, Yasin Abbasi, et al.
Published: (2024)

TBDFiltering: Sample-Efficient Tree-Based Data Filtering
by: Busa-Fekete, Robert Istvan, et al.
Published: (2026)

Faster WIND: Accelerating Iterative Best-of-$N$ Distillation for LLM Alignment
by: Yang, Tong, et al.
Published: (2024)

Understanding In-context Learning of Addition via Activation Subspaces
by: Hu, Xinyan, et al.
Published: (2025)

Managing Temporal Resolution in Continuous Value Estimation: A Fundamental Trade-off
by: Zhang, Zichen, et al.
Published: (2022)

Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
by: Galashov, Alexandre, et al.
Published: (2024)

Representation Learning via Non-Contrastive Mutual Information
by: Guo, Zhaohan Daniel, et al.
Published: (2025)

Enumerating the k-fold configurations in multi-class classification problems
by: Fazekas, Attila, et al.
Published: (2024)

Scalable Diffusion for Materials Generation
by: Yang, Sherry, et al.
Published: (2023)

Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective
by: Ghosh, Bishwamittra, et al.
Published: (2026)

Rethinking the Global Convergence of Softmax Policy Gradient with Linear Function Approximation
by: Lin, Max Qiushi, et al.
Published: (2025)

Towards Understanding Feature Learning in Parameter Transfer
by: Yuan, Hua, et al.
Published: (2025)

Towards Understanding Transformers in Learning Random Walks
by: Shi, Wei, et al.
Published: (2025)

Offline-to-online Reinforcement Learning for Image-based Grasping with Scarce Demonstrations
by: Chan, Bryan, et al.
Published: (2024)

Frontier LLMs Still Struggle with Simple Reasoning Tasks
by: Malek, Alan, et al.
Published: (2025)

Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment
by: Wilcoxson, Max, et al.
Published: (2024)

Understanding and Improving Continuous Adversarial Training for LLMs via In-context Learning Theory
by: Fu, Shaopeng, et al.
Published: (2026)

Distributed Detection of Adversarial Attacks in Multi-Agent Reinforcement Learning with Continuous Action Space
by: Kazari, Kiarash, et al.
Published: (2025)

SmallWorlds: Assessing Dynamics Understanding of World Models in Isolated Environments
by: Li, Xinyi, et al.
Published: (2025)