Image de couverture de livre

A Scalable Measure of Loss Landscape Curvature for Analyzing the Training Dynamics of LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Kalra, Dayal Singh, Gagnon-Audet, Jean-Christophe, Gromov, Andrey, Mediratta, Ishita, Niu, Kelvin, Miller, Alexander H, Shvartsman, Michael
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Disordered Systems and Neural Networks Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Why Warmup the Learning Rate? Underlying Mechanisms and Improvements
par: Kalra, Dayal Singh, et autres
Publié: (2024)

Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos
par: Kalra, Dayal Singh, et autres
Publié: (2023)

Quantifying Hyperparameter Transfer and the Importance of Embedding Layer Learning Rate
par: Kalra, Dayal Singh, et autres
Publié: (2026)

When Can You Get Away with Low Memory Adam?
par: Kalra, Dayal Singh, et autres
Publié: (2025)

(How) Can Transformers Predict Pseudo-Random Numbers?
par: Tao, Tao, et autres
Publié: (2025)

To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasets
par: Doshi, Darshil, et autres
Publié: (2023)

On the origin of neural scaling laws: from random graphs to natural language
par: Barkeshli, Maissam, et autres
Publié: (2026)

Towards Distributed Neural Architectures
par: Cowsik, Aditya, et autres
Publié: (2025)

Color symmetry breaking in the Potts spin glass
par: Mourrat, Jean-Christophe
Publié: (2024)

Spin glasses and the Parisi formula
par: Mourrat, Jean-Christophe
Publié: (2025)

Exploring Loss Landscapes through the Lens of Spin Glass Theory
par: Liao, Hao, et autres
Publié: (2024)

Inhomogeneous Floquet thermalization
par: Bera, Soumya, et autres
Publié: (2024)

An informal introduction to the Parisi formula
par: Mourrat, Jean-Christophe
Publié: (2024)

Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks
par: He, Tianyu, et autres
Publié: (2024)

Parity-dependent reentrant topology in a Su--Schrieffer--Heeger chain with power-law quasiperiodic modulation
par: Niu, Yusheng, et autres
Publié: (2026)

Thermodynamic Curvature and the Widom Ridge in Interacting Spin Systems
par: Bittner, Eric R.
Publié: (2026)

Relationship between the shear moduli and defect-induced structural relaxation ofhigh-entropy metallic glasses
par: Makarov, Andrey, et autres
Publié: (2025)

Direct Measurement of the Effective Electronic Temperature in Organic Semiconductors
par: Kompatscher, Anton, et autres
Publié: (2025)

Depinning of KPZ Interfaces in Fractional Brownian Landscapes
par: Valizadeh, Neda, et autres
Publié: (2025)

Testing the Localization Landscape Theory on the Bethe Lattice
par: Tonetti, Lorenzo, et autres
Publié: (2025)

How to Train an Oscillator Ising Machine using Equilibrium Propagation
par: Gower, Alex
Publié: (2025)

Why is the Dynamics of Glasses Super-Arrhenius?
par: Bouchaud, Jean-Philippe
Publié: (2024)

Grokking Modular Polynomials
par: Doshi, Darshil, et autres
Publié: (2024)

Effects of Disorder on the Energy Landscape and Motional Mechanisms Involved in Lithium Ion Dynamics and Transport in Solid Electrolytes: Li5.5PS4.5Cl1.5 Argyrodite as a Case Study
par: Badragheh, Mohammad Ali, et autres
Publié: (2025)

FlowEqProp: Training Flow Matching Generative Models with Gradient Equilibrium Propagation
par: Gower, Alex
Publié: (2026)

Dynamical localization and slow dynamics in quasiperiodically-driven quantum systems
par: Tiwari, Vatsana, et autres
Publié: (2023)

Disordered harmonic chain with random masses and springs: a combinatorial approach
par: Bernard, Maximilien, et autres
Publié: (2025)

Quantum transport under oscillatory drive with disordered amplitude
par: Tiwari, Vatsana, et autres
Publié: (2024)

Mean-squared Energy Difference for Exploring Potential Energy Landscapes of Supercooled Liquids
par: Zhang, Dianmo, et autres
Publié: (2025)

Reshaping Global Loop Structure to Accelerate Local Optimization by Smoothing Rugged Landscapes
par: Leleu, Timothee, et autres
Publié: (2026)

Scalable Training of Neural Network Potentials for Complex Interfaces Through Data Augmentation
par: Yeu, In Won, et autres
Publié: (2024)

The Spectral Boundary of Block Structured Random Matrices
par: Patil, Nirbhay, et autres
Publié: (2023)

Rare Trajectories in a Prototypical Mean-field Disordered Model: Insights into Landscape and Instantons
par: Charbonneau, Patrick, et autres
Publié: (2025)

Machine Learning the Disorder Landscape of Majorana Nanowires
par: Taylor, Jacob R., et autres
Publié: (2023)

Exploring the Energy Landscape of RBMs: Reciprocal Space Insights into Bosons, Hierarchical Learning and Symmetry Breaking
par: Toledo-Marin, J. Quetzalcóatl, et autres
Publié: (2025)

Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval
par: Maillard, Antoine, et autres
Publié: (2026)

Classical analog circuit emulation of quantum Grover search algorithm
par: Feldman, Samuel, et autres
Publié: (2025)

Absence of Weak Localization on Negative Curvature Surfaces
par: Curtis, Jonathan B., et autres
Publié: (2023)

Minimalistic and Scalable Quantum Reservoir Computing Enhanced with Feedback
par: Zhu, Chuanzhou, et autres
Publié: (2024)

Non-isothermal flow of Al-, Co- and Cu-based alloys made in different spatial configurations or structural states: model and experimental study
par: Berezner, A. D., et autres
Publié: (2026)