The Persistence of Neural Collapse Despite Low-Rank Bias

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Garrod, Connall, Keating, Jonathan P.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Unifying Low Dimensional Spectra in Deep Learning
by: Garrod, Connall, et al.
Published: (2024)

Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
by: Garrod, Connall, et al.
Published: (2025)

Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?
by: Súkeník, Peter, et al.
Published: (2024)

Provable Emergence of Deep Neural Collapse and Low-Rank Bias in $L^2$-Regularized Nonlinear Networks
by: Zangrando, Emanuele, et al.
Published: (2024)

Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse
by: Baker, Bradley T., et al.
Published: (2024)

The Inlet Rank Collapse in Implicit Neural Representations: Diagnosis and Unified Remedy
by: Zheng, Jianqiao, et al.
Published: (2026)

On the Robustness of Neural Collapse and the Neural Collapse of Robustness
by: Su, Jingtong, et al.
Published: (2023)

Preventing Rank Collapse in Federated Low-Rank Adaptation with Client Heterogeneity
by: Wu, Fei, et al.
Published: (2026)

Low-Rank Adaptation of Neural Fields
by: Truong, Anh, et al.
Published: (2025)

Implicit Bias and Loss of Plasticity in Matrix Completion: Depth Promotes Low-Rankness
by: Shin, Baekrok, et al.
Published: (2026)

Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networks
by: Roth, Andreas, et al.
Published: (2023)

The Loss Surfaces of Neural Networks with General Activation Functions
by: Baskerville, Nicholas P., et al.
Published: (2020)

The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions
by: Schmidt, Jonathan, et al.
Published: (2023)

On Generalization Bounds for Neural Networks with Low Rank Layers
by: Pinto, Andrea, et al.
Published: (2024)

Latent Point Collapse on a Low Dimensional Embedding in Deep Neural Network Classifiers
by: Sbailò, Luigi, et al.
Published: (2023)

Lambda-Skip Connections: the architectural component that prevents Rank Collapse
by: Joseph, Federico Arangath, et al.
Published: (2024)

Preventing Representational Rank Collapse in MPNNs by Splitting the Computational Graph
by: Roth, Andreas, et al.
Published: (2024)

Low-Rank Prehab: Preparing Neural Networks for SVD Compression
by: Qin, Haoran, et al.
Published: (2025)

Personalizing Low-Rank Bayesian Neural Networks Via Federated Learning
by: Zhang, Boning, et al.
Published: (2024)

Simple yet Effective: Low-Rank Spatial Attention for Neural Operators
by: Yang, Zherui, et al.
Published: (2026)

Harnessing Orthogonality to Train Low-Rank Neural Networks
by: Coquelin, Daniel, et al.
Published: (2024)

Low-Rank Tensor Decompositions for the Theory of Neural Networks
by: Borsoi, Ricardo, et al.
Published: (2025)

Low-Rank Matrix Approximation for Neural Network Compression
by: Cherukuri, Kalyan, et al.
Published: (2025)

AI Control: Improving Safety Despite Intentional Subversion
by: Greenblatt, Ryan, et al.
Published: (2023)

Stability for Inference with Persistent Homology Rank Functions
by: Wang, Qiquan, et al.
Published: (2023)

InRank: Incremental Low-Rank Learning
by: Zhao, Jiawei, et al.
Published: (2023)

A Relational Inductive Bias for Dimensional Abstraction in Neural Networks
by: Campbell, Declan, et al.
Published: (2024)

Only Large Weights (And Not Skip Connections) Can Prevent the Perils of Rank Collapse
by: Alman, Josh, et al.
Published: (2025)

Low Rank Based Subspace Inference for the Laplace Approximation of Bayesian Neural Networks
by: Faller, Josua, et al.
Published: (2025)

Persistence-Augmented Neural Networks
by: Wang, Elena Xinyi, et al.
Published: (2026)

Sebra: Debiasing Through Self-Guided Bias Ranking
by: Kappiyath, Adarsh, et al.
Published: (2025)

Theoretical Guarantees for Low-Rank Compression of Deep Neural Networks
by: Zhang, Shihao, et al.
Published: (2025)

Pushing Boundaries: Mixup's Influence on Neural Collapse
by: Fisher, Quinn, et al.
Published: (2024)

Rethinking Continual Learning with Progressive Neural Collapse
by: Wang, Zheng, et al.
Published: (2025)

Optimizer choice matters for the emergence of Neural Collapse
by: Zhao, Jim, et al.
Published: (2026)

Trojan Cleansing with Neural Collapse
by: Gu, Xihe, et al.
Published: (2024)

Towards Better Generalization: Weight Decay Induces Low-rank Bias for Neural Networks
by: Chen, Ke, et al.
Published: (2024)

FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks
by: Sun, Jialong, et al.
Published: (2025)

The Prevalence of Neural Collapse in Neural Multivariate Regression
by: Andriopoulos, George, et al.
Published: (2024)

Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers
by: Saada, Thiziri Nait, et al.
Published: (2024)