Image de couverture de livre

Equivalence of Context and Parameter Updates in Modern Transformer Blocks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Goldwaser, Adrian, Munn, Michael, Gonzalvo, Javier, Dherin, Benoit
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

A Margin-based Multiclass Generalization Bound via Geometric Complexity
par: Munn, Michael, et autres
Publié: (2024)

The Impact of Geometric Complexity on Neural Collapse in Transfer Learning
par: Munn, Michael, et autres
Publié: (2024)

Transmuting prompts into weights
par: Mazzawi, Hanna, et autres
Publié: (2025)

On residual network depth
par: Dherin, Benoit, et autres
Publié: (2025)

Learning without training: The implicit dynamics of in-context learning
par: Dherin, Benoit, et autres
Publié: (2025)

Learning by solving differential equations
par: Dherin, Benoit, et autres
Publié: (2025)

How iteration order influences convergence and stability in deep learning
par: Dherin, Benoit, et autres
Publié: (2025)

Grow, Don't Overwrite: Fine-tuning Without Forgetting
par: Adila, Dyah, et autres
Publié: (2026)

Deep Fusion: Efficient Network Training via Pre-trained Initializations
par: Mazzawi, Hanna, et autres
Publié: (2023)

Corridor Geometry in Gradient-Based Optimization
par: Dherin, Benoit, et autres
Publié: (2024)

A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints
par: Munn, Michael, et autres
Publié: (2024)

How Data Mixing Shapes In-Context Learning: Asymptotic Equivalence for Transformers with MLPs
par: Demir, Samet, et autres
Publié: (2025)

In-Context In-Context Learning with Transformer Neural Processes
par: Ashman, Matthew, et autres
Publié: (2024)

The Update-Equivalence Framework for Decision-Time Planning
par: Sokota, Samuel, et autres
Publié: (2023)

Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix
par: Hayase, Tomohiro, et autres
Publié: (2025)

Simplifying Transformer Blocks
par: He, Bobby, et autres
Publié: (2023)

Convergence for Discrete Parameter Update Schemes
par: Wilson, Paul, et autres
Publié: (2025)

Circuit Transformer: A Transformer That Preserves Logical Equivalence
par: Li, Xihan, et autres
Publié: (2024)

Towards Understanding the Link Between Modularity and Performance in Neural Networks for Reinforcement Learning
par: Munn, Humphrey, et autres
Publié: (2022)

Partial Parameter Updates for Efficient Distributed Training
par: Filippova, Anastasiia, et autres
Publié: (2025)

Allocation of Parameters in Transformers
par: Yu, Ruoxi, et autres
Publié: (2025)

Gradient Transformer: Learning to Generate Updates for LLMs
par: Nguyen, Binh-Nguyen, et autres
Publié: (2026)

Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learning
par: He, JinLi, et autres
Publié: (2026)

Practical and Optimal Algorithm for Linear Contextual Bandits with Rare Parameter Updates
par: Yu, Sanghoon, et autres
Publié: (2026)

In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization
par: Zhang, Ruiqi, et autres
Publié: (2024)

Majority Kernels: An Approach to Leverage Big Model Dynamics for Efficient Small Model Training
par: Mazzawi, Hanna, et autres
Publié: (2024)

Aggregation Buffer: Revisiting DropEdge with a New Parameter Block
par: Lee, Dooho, et autres
Publié: (2025)

Domain Expansion: Parameter-Efficient Modules as Building Blocks for Composite Domains
par: Patel, Mann, et autres
Publié: (2025)

PETAH: Parameter Efficient Task Adaptation for Hybrid Transformers in a resource-limited Context
par: Augustin, Maximilian, et autres
Publié: (2024)

CART: Context-Anchored Recurrent Transformer -- A Parameter-Efficient Architecture with Learned Stability
par: Capps, Chad A.
Publié: (2026)

Asymptotic Study of In-context Learning with Random Transformers through Equivalent Models
par: Demir, Samet, et autres
Publié: (2025)

Cluster Purge Loss: Structuring Transformer Embeddings for Equivalent Mutants Detection
par: Danilov, Adelaide, et autres
Publié: (2025)

Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
par: Shi, Kexuan, et autres
Publié: (2026)

Free Random Projection for In-Context Reinforcement Learning
par: Hayase, Tomohiro, et autres
Publié: (2025)

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
par: Guo, Han, et autres
Publié: (2026)

How do Transformers perform In-Context Autoregressive Learning?
par: Sander, Michael E., et autres
Publié: (2024)

Secure Generalization through Stochastic Bidirectional Parameter Updates Using Dual-Gradient Mechanism
par: Goel, Shourya, et autres
Publié: (2025)

Transformer-based Parameter Estimation in Statistics
par: Yin, Xiaoxin, et autres
Publié: (2024)

RT-Transformer: The Transformer Block as a Spherical State Estimator
par: Racioppo, Peter
Publié: (2026)

Composite Learning Units: Generalized Learning Beyond Parameter Updates to Transform LLMs into Adaptive Reasoners
par: Radha, Santosh Kumar, et autres
Publié: (2024)