Image de couverture de livre

u-$μ$P: The Unit-Scaled Maximal Update Parametrization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Blake, Charlie, Eichenberg, Constantin, Dean, Josef, Balles, Lukas, Prince, Luke Y., Deiseroth, Björn, Cruz-Salinas, Andres Felipe, Luschi, Carlo, Weinbach, Samuel, Orr, Douglas
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models
par: Neitemeier, Pit, et autres
Publié: (2025)

SparQ Attention: Bandwidth-Efficient LLM Inference
par: Ribar, Luka, et autres
Publié: (2023)

Optimal Formats for Weight Quantisation
par: Orr, Douglas, et autres
Publié: (2025)

T-FREE: Subword Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings
par: Deiseroth, Björn, et autres
Publié: (2024)

1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization
par: Maskey, Sohir, et autres
Publié: (2026)

Divergent Token Metrics: Measuring degradation to prune away LLM components -- and optimize quantization
par: Deiseroth, Björn, et autres
Publié: (2023)

AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
par: Deiseroth, Björn, et autres
Publié: (2023)

MXNorm: Reusing MXFP block scales for efficient tensor normalisation
par: McLean, Callum, et autres
Publié: (2026)

Elucidating the Design Space of FP4 training
par: Hu, Robert, et autres
Publié: (2025)

Ground-Truth Subgraphs for Better Training and Evaluation of Knowledge Graph Augmented LLMs
par: Cattaneo, Alberto, et autres
Publié: (2025)

Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need
par: Wistuba, Martin, et autres
Publié: (2024)

When the Whole Is Less Than the Sum of Its Parts: Structural Coupling in Education
par: Raf Vanderstraeten, et autres
Publié: (2026)

Bounding Hallucinations: Information-Theoretic Guarantees for RAG Systems via Merlin-Arthur Protocols
par: Deiseroth, Björn, et autres
Publié: (2025)

AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency
par: Höth, Max Henning, et autres
Publié: (2026)

Scalify: scale propagation for efficient low-precision LLM training
par: Balança, Paul, et autres
Publié: (2024)

Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators
par: Li, Shanda, et autres
Publié: (2025)

$μ$-Parametrization for Mixture of Experts
par: Małaśnicki, Jan, et autres
Publié: (2025)

A Family of LLMs Liberated from Static Vocabularies
par: Alpha, Aleph, et autres
Publié: (2026)

Chapter Building’s Twin Reconstruction
par: Luschi, Cecilia, et autres
Publié: (2024)

Chapter Le rovine del Sant’Anna a Beit Guvrin e l’esportazione di protocolli geometrici per la costruzione
par: Luschi, Cecilia, et autres
Publié: (2025)

LIME: Making LLM Data More Efficient with Linguistic Metadata Embeddings
par: Sztwiertnia, Sebastian, et autres
Publié: (2025)

Rings with $u-1$ Quasinilpotent for Each Unit $u$
par: Danchev, Peter, et autres
Publié: (2024)

Rings With $u^n-1$ Nilpotent For Each Unit $u$
par: Danchev, Peter, et autres
Publié: (2023)

Rings Such That $u-1$ Lies In $J^{\#}(R)$ For Each Unit $u$
par: Danchev, Peter, et autres
Publié: (2025)

SCAR: Sparse Conditioned Autoencoders for Concept Detection and Steering in LLMs
par: Härle, Ruben, et autres
Publié: (2024)

Efficient Parallelization Layouts for Large-Scale Distributed Model Training
par: Hagemann, Johannes, et autres
Publié: (2023)

SOMBRERO: Measuring and Steering Boundary Placement in End-to-End Hierarchical Sequence Models
par: Neitemeier, Pit, et autres
Publié: (2026)

The Winnability of Klondike Solitaire and Many Other Patience Games
par: Blake, Charlie, et autres
Publié: (2019)

Maximal spreading of impacting viscoelastic droplets
par: Avni, Orr, et autres
Publié: (2026)

UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG
par: Georgiev, Dobrik, et autres
Publié: (2026)

Measuring and Guiding Monosemanticity
par: Härle, Ruben, et autres
Publié: (2025)

The Paradoxical Effect of Repeated Body Checking on Subjective Uncertainty
par: Mor Ben Zaken Linn, et autres
Publié: (2024)

Spatz: Clustering Compact RISC-V-Based Vector Units to Maximize Computing Efficiency
par: Perotti, Matteo, et autres
Publié: (2023)

Approximate Top-$k$ for Increased Parallelism
par: Key, Oscar, et autres
Publié: (2024)

Updates on transversity extractions
par: Flore, Carlo
Publié: (2024)

From a P‐Bridging Phosphaketene to μ‐Phosphinidenide and μ‐Diphosphaurea Units at a Dinickel Core
par: Roland A. Schulz, et autres
Publié: (2024)

How the Cookie Crumbles: A Model for Star-forming Clumps in High-redshift Disk Galaxies
par: Orr, Matthew E., et autres
Publié: (2024)

Diffusion-Driven Generation of Minimally Preprocessed Brain MRI
par: Remedios, Samuel W., et autres
Publié: (2025)

Higgs boson production at $μ^+ μ^+$ colliders
par: Hamada, Yu, et autres
Publié: (2024)

bsiepe/Dissertation: v.1.1.0: Updated Figures
par: Björn Siepe
Publié: (2026)