Cover Image

The Spectral Lifecycle of Transformer Training: Transient Compression Waves, Persistent Spectral Gradients, and the Q/K--V Asymmetry

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Liu, Yi
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Graph Persistence goes Spectral
by: Ji, Mattie, et al.
Published: (2025)

Residual Koopman Spectral Profiling for Predicting and Preventing Transformer Training Instability
by: Kim, Bum Jun, et al.
Published: (2026)

Controlled LLM Training on Spectral Sphere
by: Xie, Tian, et al.
Published: (2026)

Spectral Transformer Neural Processes
by: Chen, Xianhe, et al.
Published: (2026)

Dive into Waves: Morlet Spectral Transformer for Cross-Subject Emotion Decoding from EEG
by: Qing, Jiaxin, et al.
Published: (2026)

Flash STU: Fast Spectral Transform Units
by: Liu, Y. Isabel, et al.
Published: (2024)

Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting
by: Tian, Zailong, et al.
Published: (2026)

The Lifecycle of the Spectral Edge: From Gradient Learning to Weight-Decay Compression
by: Xu, Yongzhong
Published: (2026)

Dynamic Spectral Backpropagation for Efficient Neural Network Training
by: Muthuraman, Mannmohan
Published: (2025)

Spectral Adapter: Fine-Tuning in Spectral Space
by: Zhang, Fangzhao, et al.
Published: (2024)

Spectral Discovery of Continuous Symmetries via Generalized Fourier Transforms
by: Karjol, Pavan, et al.
Published: (2026)

EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training
by: Yi, Qingao, et al.
Published: (2025)

Spectral Geometry for Deep Learning: Compression and Hallucination Detection via Random Matrix Theory
by: Ettori, Davide
Published: (2026)

Spectral bandits
by: Kocák, Tomáš, et al.
Published: (2026)

Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology
by: Fernando, Jesseba, et al.
Published: (2026)

Anytime Training with Schedule-Free Spectral Optimization
by: Apte, Anuj, et al.
Published: (2026)

The Fourier Spectral Transformer Networks For Efficient and Generalizable Nonlinear PDEs Prediction
by: Li, Beibei
Published: (2025)

Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training
by: Emadi, Seyed Morteza
Published: (2026)

Spectral Edge Dynamics of Training Trajectories: Signal--Noise Geometry Across Scales
by: Xu, Yongzhong
Published: (2026)

Spectral Gating Networks
by: Zhang, Jusheng, et al.
Published: (2026)

Fast Fourier Transform-Based Spectral and Temporal Gradient Filtering for Differential Privacy
by: Shin, Hyeju, et al.
Published: (2025)

Airport Passenger Flow Forecasting via Deformable Temporal-Spectral Transformer Approach
by: Du, Wenbo, et al.
Published: (2025)

SVDformer: Direction-Aware Spectral Graph Embedding Learning via SVD and Transformer
by: Fang, Jiayu, et al.
Published: (2025)

On Spectral Properties of Gradient-based Explanation Methods
by: Mehrpanah, Amir, et al.
Published: (2025)

Zero-Sacrifice Persistent-Robustness Adversarial Defense for Pre-Trained Encoders
by: Lei, Zhuxin, et al.
Published: (2026)

Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent
by: Naganuma, Hiroki, et al.
Published: (2026)

Spectral Edge Dynamics: An Analytical-Empirical Study of Phase Transitions in Neural Network Training
by: Xu, Yongzhong
Published: (2026)

Rethinking Spectral Graph Neural Networks with Spatially Adaptive Filtering
by: Guo, Jingwei, et al.
Published: (2024)

The Homogeneity Trap: Spectral Collapse in Doubly-Stochastic Deep Networks
by: Liu, Yizhi
Published: (2026)

Spectral Clustering for Discrete Distributions
by: Wang, Zixiao, et al.
Published: (2024)

Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations
by: Acharya, Pratyush, et al.
Published: (2026)

STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem
by: Wang, Hong, et al.
Published: (2025)

Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization
by: Yi, Kai
Published: (2025)

Multi-Dimensional Spectral Geometry of Biological Knowledge in Single-Cell Transformer Representations
by: Kendiukhov, Ihor
Published: (2026)

Beyond the Birkhoff Polytope: Spectral-Sphere-Constrained Hyper-Connections
by: Liu, Zhaoyi, et al.
Published: (2026)

Spectral Higher-Order Neural Networks
by: Peri, Gianluca, et al.
Published: (2026)

Diffusion Spectral Representation for Reinforcement Learning
by: Shribak, Dmitry, et al.
Published: (2024)

Spectral Representation-based Reinforcement Learning
by: Gao, Chenxiao, et al.
Published: (2025)

Spatio-Spectral Graph Neural Networks
by: Geisler, Simon, et al.
Published: (2024)

Shape-aware Graph Spectral Learning
by: Xu, Junjie, et al.
Published: (2023)