The Geometry of Projection Heads: Conditioning, Invariance, and Collapse

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Chaudhry, Faris
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Trajectory-Restricted Optimization Conditions and Geometry-Aware Linear Convergence
by: Chaudhry, Faris, et al.
Published: (2026)

Asymptotic and Finite-Time Guarantees for Langevin-Based Temperature Annealing in InfoNCE
by: Chaudhry, Faris
Published: (2026)

Feature Starvation as Geometric Instability in Sparse Autoencoders
by: Chaudhry, Faris, et al.
Published: (2026)

Higher-Order Newton Methods with Polynomial Work per Iteration
by: Ahmadi, Amir Ali, et al.
Published: (2023)

Forward Invariance in Neural Network Controlled Systems
by: Harapanahalli, Akash, et al.
Published: (2023)

Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?
by: Súkeník, Peter, et al.
Published: (2024)

The Implicit Bias of Heterogeneity towards Invariance: A Study of Multi-Environment Matrix Sensing
by: Xu, Yang, et al.
Published: (2024)

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
by: Jacot, Arthur, et al.
Published: (2024)

Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data
by: Min, Hancheng, et al.
Published: (2025)

Corridor Geometry in Gradient-Based Optimization
by: Dherin, Benoit, et al.
Published: (2024)

Safely Learning Dynamical Systems
by: Ahmadi, Amir Ali, et al.
Published: (2023)

Separating Geometry from Probability in the Analysis of Generalization
by: Raginsky, Maxim, et al.
Published: (2026)

MDP Geometry, Normalization and Reward Balancing Solvers
by: Mustafin, Arsenii, et al.
Published: (2024)

Monotone Optimisation with Learned Projections
by: Rashwan, Ahmed, et al.
Published: (2026)

Set Invariance with Probability One for Controlled Diffusion: Score-based Approach
by: Wang, Wenqing, et al.
Published: (2025)

Geometry-Preserving Neural Architectures on Manifolds with Boundary
by: Elamvazhuthi, Karthik, et al.
Published: (2026)

PAC Learnability of Scenario Decision-Making Algorithms: Necessary Conditions and Sufficient Conditions
by: Berger, Guillaume O., et al.
Published: (2025)

Multistage Conditional Compositional Optimization
by: Şen, Buse, et al.
Published: (2026)

Adaptive Conditional Gradient Descent
by: Khademi, Abbas, et al.
Published: (2025)

A Precise Characterization of SGD Stability Using Loss Surface Geometry
by: Dexter, Gregory, et al.
Published: (2024)

Second-order Conditional Gradient Sliding
by: Carderera, Alejandro, et al.
Published: (2020)

Robust Sublinear Convergence Rates for Iterative Bregman Projections
by: Peyré, Gabriel
Published: (2026)

Optimal Projection-Free Adaptive SGD for Matrix Optimization
by: Kovalev, Dmitry
Published: (2026)

Improving Feasibility via Fast Autoencoder-Based Projections
by: Chzhen, Maria, et al.
Published: (2026)

Mitigating Forgetting in Continual Learning with Selective Gradient Projection
by: Singh, Anika, et al.
Published: (2026)

Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise
by: Zhang, Jiayu, et al.
Published: (2026)

SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions
by: Liu, Kang, et al.
Published: (2026)

Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project
by: Rathore, Pratik, et al.
Published: (2025)

On the Inherent Privacy of Zeroth Order Projected Gradient Descent
by: Gupta, Devansh, et al.
Published: (2025)

Riemannian Optimization for Non-convex Euclidean Distance Geometry with Global Recovery Guarantees
by: Smith, Chandler, et al.
Published: (2024)

Non-Euclidean Broximal Point Method: A Blueprint for Geometry-Aware Optimization
by: Gruntkowska, Kaja, et al.
Published: (2025)

Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networks
by: Zhang, Leyang, et al.
Published: (2024)

Natural Geometry of Robust Data Attribution: From Convex Models to Deep Networks
by: Li, Shihao, et al.
Published: (2025)

Causal Invariance Learning via Efficient Nonconvex Optimization
by: Wang, Zhenyu, et al.
Published: (2024)

Projection-Free Online Convex Optimization with Time-Varying Constraints
by: Garber, Dan, et al.
Published: (2024)

Robust Multi-Dimensional Scaling via Accelerated Alternating Projections
by: Deng, Tong, et al.
Published: (2025)

Universal Online Convex Optimization with $1$ Projection per Round
by: Yang, Wenhao, et al.
Published: (2024)

A Sketch-and-Project Analysis of Subsampled Natural Gradient Algorithms
by: Goldshlager, Gil, et al.
Published: (2025)

Projection-free Online Learning over Strongly Convex Sets
by: Wan, Yuanyu, et al.
Published: (2020)

On the Role of Batch Size in Stochastic Conditional Gradient Methods
by: Islamov, Rustem, et al.
Published: (2026)