Image de couverture de livre

Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liu, Andy Zeyi, Paquette, Elliot, Sous, John
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Memory Inception: Latent-Space KV Cache Manipulation for Steering LLMs
par: Liu, Andy Zeyi, et autres
Publié: (2026)

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions
par: Marshall, Noah, et autres
Publié: (2024)

Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling
par: Benigni, Lucas, et autres
Publié: (2025)

High-Dimensional Privacy-Utility Dynamics of Noisy Stochastic Gradient Descent on Least Squares
par: Lin, Shurong, et autres
Publié: (2025)

Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
par: Everett, Katie, et autres
Publié: (2026)

4+3 Phases of Compute-Optimal Neural Scaling Laws
par: Paquette, Elliot, et autres
Publié: (2024)

Logarithmic-time Schedules for Scaling Language Models with Momentum
par: Ferbach, Damien, et autres
Publié: (2026)

Dimension-adapted Momentum Outscales SGD
par: Ferbach, Damien, et autres
Publié: (2025)

Power-Law Spectrum of the Random Feature Model
par: Paquette, Elliot, et autres
Publié: (2026)

Assign and Add: A Mechanistic Study of Compositional Arithmetic
par: Exoo, Brady, et autres
Publié: (2026)

Phases of Muon: When Muon Eclipses SignSGD
par: Paquette, Elliot, et autres
Publié: (2026)

Asymmetric Scaling Laws from Sparse Features
par: Sous, John, et autres
Publié: (2026)

Exact Risk Curves of signSGD in High-Dimensions: Quantifying Preconditioning and Noise-Compression Effects
par: Xiao, Ke Liang, et autres
Publié: (2024)

Anisotropic local law for non-separable sample covariance matrices
par: Fan, Zhou, et autres
Publié: (2026)

Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropy
par: Huang, Zhendong, et autres
Publié: (2026)

The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms
par: Collins-Woodfin, Elizabeth, et autres
Publié: (2024)

Fitting an ellipsoid to a quadratic number of random points
par: Bandeira, Afonso S., et autres
Publié: (2023)

Muon in Vision Transformers: Optimizer-Recipe Interactions and Gradient Spectra
par: Southworth, Ben S., et autres
Publié: (2026)

SpectraKAN: Conditioning Spectral Operators
par: Cheng, Chun-Wun, et autres
Publié: (2026)

LossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics
par: Xie, Tiankai, et autres
Publié: (2024)

Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance
par: Kaushik, Chiraag, et autres
Publié: (2024)

(Im)possibility of Automated Hallucination Detection in Large Language Models
par: Karbasi, Amin, et autres
Publié: (2025)

GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
par: Chen, Tianhao, et autres
Publié: (2025)

EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization
par: Song, Zhiye, et autres
Publié: (2026)

IMPROVE: Iterative Model Pipeline Refinement and Optimization Leveraging LLM Experts
par: Xue, Eric, et autres
Publié: (2025)

Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning
par: Lu, Binghang, et autres
Publié: (2026)

Correctness-Optimized Residual Activation Lens (CORAL): Transferrable and Calibration-Aware Inference-Time Steering
par: Miao, Miranda Muqing, et autres
Publié: (2026)

Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distance
par: Li, Muyang, et autres
Publié: (2026)

Distributional Spectral Diagnostics for Localizing Grokking Transitions
par: Wang, Ziyue, et autres
Publié: (2026)

Out of the Ordinary: Spectrally Adapting Regression for Covariate Shift
par: Eyre, Benjamin, et autres
Publié: (2023)

Revisiting LoRA through the Lens of Parameter Redundancy: Spectral Encoding Helps
par: Cheng, Jiashun, et autres
Publié: (2025)

Learning Physical Simulation with Message Passing Transformer
par: Xu, Zeyi, et autres
Publié: (2024)

Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
par: Atanasov, Alexander, et autres
Publié: (2025)

A constraints-based approach to fully interpretable neural networks for detecting learner behaviors
par: Pinto, Juan D., et autres
Publié: (2025)

Towards a Unified Framework for Evaluating Explanations
par: Pinto, Juan D., et autres
Publié: (2024)

AWP: Activation-Aware Weight Pruning and Quantization with Projected Gradient Descent
par: Liu, Jing, et autres
Publié: (2025)

Deep Learning for Optical Misalignment Diagnostics in Multi-Lens Imaging Systems
par: Slor, Tomer, et autres
Publié: (2025)

The Spectral Lifecycle of Transformer Training: Transient Compression Waves, Persistent Spectral Gradients, and the Q/K--V Asymmetry
par: Liu, Yi
Publié: (2026)

Bridging Spectral Operator Learning and U-Net Hierarchies: SpectraNet for Stable Autoregressive PDE Surrogates
par: Noguera, Enrique Hernández, et autres
Publié: (2026)

An Explainable Diagnostic Framework for Neurodegenerative Dementias via Reinforcement-Optimized LLM Reasoning
par: Zamai, Andrew, et autres
Publié: (2025)