Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Súkeník, Peter, Mondelli, Marco, Lampert, Christoph
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
by: Jacot, Arthur, et al.
Published: (2024)

Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers
by: Súkeník, Peter, et al.
Published: (2025)

Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data
by: Min, Hancheng, et al.
Published: (2025)

The Exploration of Neural Collapse under Imbalanced Data
by: Liu, Haixia
Published: (2024)

The Geometry of Projection Heads: Conditioning, Invariance, and Collapse
by: Chaudhry, Faris
Published: (2026)

Can Learning Be Explained By Local Optimality In Robust Low-rank Matrix Recovery?
by: Ma, Jianhao, et al.
Published: (2023)

Optimal Depth of Neural Networks
by: Qi, Qian
Published: (2025)

Riemannian Neural Optimal Transport
by: Micheli, Alessandro, et al.
Published: (2026)

Statistically Optimal K-means Clustering via Nonnegative Low-rank Semidefinite Programming
by: Zhuang, Yubo, et al.
Published: (2023)

Optimality-Informed Neural Networks for Solving Parametric Optimization Problems
by: Hoffmann, Matthias K., et al.
Published: (2025)

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
by: Cai, Yuhang, et al.
Published: (2025)

SGD with Partial Hessian for Deep Neural Networks Optimization
by: Sun, Ying, et al.
Published: (2024)

Deep Operator Neural Network Model Predictive Control
by: de Jong, Thomas Oliver, et al.
Published: (2025)

Efficient Duple Perturbation Robustness in Low-rank MDPs
by: Hu, Yang, et al.
Published: (2024)

Improved Scalable Lipschitz Bounds for Deep Neural Networks
by: Syed, Usman, et al.
Published: (2025)

Beyond the Neural Fog: Interpretable Learning for AC Optimal Power Flow
by: Pineda, Salvador, et al.
Published: (2024)

Compression-aware Training of Neural Networks using Frank-Wolfe
by: Zimmer, Max, et al.
Published: (2022)

Neural Approximators for Low-Thrust Trajectory Transfer Cost and Reachability
by: Zhang, Zhong, et al.
Published: (2025)

Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks
by: Qin, Zhen, et al.
Published: (2023)

Reward Collapse in Aligning Large Language Models
by: Song, Ziang, et al.
Published: (2023)

State-Space NTK Collapse Near Bifurcations
by: Hazelden, James, et al.
Published: (2026)

Optimal Potential Shaping on SE(3) via Neural ODEs on Lie Groups
by: Wotte, Yannik P., et al.
Published: (2024)

Calibrating Neural Networks' parameters through Optimal Contraction in a Prediction Problem
by: Gonzalo, Valdes
Published: (2024)

Towards Optimal Branching of Linear and Semidefinite Relaxations for Neural Network Robustness Certification
by: Anderson, Brendon G., et al.
Published: (2021)

A Guaranteed-Stable Neural Network Approach for Optimal Control of Nonlinear Systems
by: Li, Anran, et al.
Published: (2025)

Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime
by: Wu, Diyuan, et al.
Published: (2025)

Exploring the Potential of Bilevel Optimization for Calibrating Neural Networks
by: Sanguin, Gabriele, et al.
Published: (2025)

Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention
by: Súkeník, Peter, et al.
Published: (2026)

Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
by: Tucat, Matteo, et al.
Published: (2024)

Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations
by: Kumar, Akshay, et al.
Published: (2024)

Tight Robustness Certificates and Wasserstein Distributional Attacks for Deep Neural Networks
by: Le, Bach C., et al.
Published: (2025)

Weight-Parameterization in Continuous Time Deep Neural Networks for Surrogate Modeling
by: Rosso, Haley, et al.
Published: (2025)

Solving Max-Cut to Global Optimality via Feasibility-Preserving Graph Neural Networks
by: Chen, Hao, et al.
Published: (2026)

In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting
by: Philippenko, Constantin, et al.
Published: (2024)

How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression?
by: Lai, Kuo-Wei, et al.
Published: (2026)

Bias-Optimal Bounds for SGD: A Computer-Aided Lyapunov Analysis
by: Cortild, Daniel, et al.
Published: (2025)

Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes
by: Zhang, Erica, et al.
Published: (2024)

CRONOS: Enhancing Deep Learning with Scalable GPU Accelerated Convex Neural Networks
by: Feng, Miria, et al.
Published: (2024)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
by: Liao, Fangshuo, et al.
Published: (2023)

Sharp Global Guarantees for Nonconvex Low-rank Recovery in the Noisy Overparameterized Regime
by: Zhang, Richard Y.
Published: (2021)