Cover Image

Network Degeneracy as an Indicator of Training Performance: Comparing Finite and Infinite Width Angle Predictions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jakub, Cameron, Nica, Mihai
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization
by: Jakub, Cameron, et al.
Published: (2023)

Differential Equation Scaling Limits of Shaped and Unshaped Neural Networks
by: Li, Mufan Bill, et al.
Published: (2023)

An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network
by: Kim, Taeyoung, et al.
Published: (2023)

Channel Estimation by Infinite Width Convolutional Networks
by: Mallik, Mohammed, et al.
Published: (2025)

Flexible Infinite-Width Graph Convolutional Neural Networks
by: Anson, Ben, et al.
Published: (2024)

Infinite Width Limits of Self Supervised Neural Networks
by: Fleissner, Maximilian, et al.
Published: (2024)

On the Infinite Width and Depth Limits of Predictive Coding Networks
by: Innocenti, Francesco, et al.
Published: (2026)

Depth Separation in Norm-Bounded Infinite-Width Neural Networks
by: Parkinson, Suzanna, et al.
Published: (2024)

Dynamic Sparse Training with Structured Sparsity
by: Lasby, Mike, et al.
Published: (2023)

Homotopy Relaxation Training Algorithms for Infinite-Width Two-Layer ReLU Neural Networks
by: Yang, Yahong, et al.
Published: (2023)

Transfer Learning in Infinite Width Feature Learning Networks
by: Lauditi, Clarissa, et al.
Published: (2025)

Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation
by: Ishikawa, Satoki, et al.
Published: (2024)

On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width
by: Ishikawa, Satoki, et al.
Published: (2023)

The Graphon Limit Hypothesis: Understanding Neural Network Pruning via Infinite Width Analysis
by: Pham, Hoang, et al.
Published: (2025)

Some Theoretical Results on Layerwise Effective Dimension Oscillations in Finite Width ReLU Networks
by: Makwana, Darshan
Published: (2025)

Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs
by: Sakai, Mana, et al.
Published: (2025)

Efficient Techniques for Data Reconstruction, with Finite-Width Recovery Guarantees
by: Tansley, Edward, et al.
Published: (2026)

Statistical Physics of Deep Neural Networks: Generalization Capability, Beyond the Infinite Width, and Feature Learning
by: Ariosto, Sebastiano
Published: (2025)

From Sublinear to Linear: Local Convergence in Finite-Width Networks via Locally Polyak-Lojasiewicz Regions
by: Aich, Agnideep, et al.
Published: (2025)

Mathematical Foundations of Neural Tangents and Infinite-Width Networks
by: Mysore, Rachana, et al.
Published: (2025)

How Long Does Infinite Width Last? Signal Propagation in Long-Range Linear Recurrences
by: Seleznova, Mariia
Published: (2026)

Virtual Width Networks
by: Seed, et al.
Published: (2025)

Comparative analysis of Realistic EMF Exposure Estimation from Low Density Sensor Network by Finite & Infinite Neural Networks
by: Mallik, Mohammed, et al.
Published: (2025)

Les Houches Lectures on Deep Learning at Large & Infinite Width
by: Bahri, Yasaman, et al.
Published: (2023)

Adaptive Width Neural Networks
by: Errica, Federico, et al.
Published: (2025)

Infinite Width Models That Work: Why Feature Learning Doesn't Matter as Much as You Think
by: Sernau, Luke
Published: (2024)

Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $μ$P Parametrization
by: Chen, Zixiang, et al.
Published: (2025)

Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks
by: Huang, Ann, et al.
Published: (2024)

Minimum Width of Deep Narrow Networks for Universal Approximation
by: Yang, Xiao-Song, et al.
Published: (2025)

Finite-Width Neural Tangent Kernels from Feynman Diagrams
by: Guillen, Max, et al.
Published: (2025)

CP Degeneracy in Tensor Regression
by: Zhou, Ya, et al.
Published: (2020)

Degeneracy is OK: Logarithmic Regret for Network Revenue Management with Indiscrete Distributions
by: Jiang, Jiashuo, et al.
Published: (2022)

The Spectral Dimension of NTKs is Constant: A Theory of Implicit Regularization, Finite-Width Stability, and Scalable Estimation
by: Shukla, Praveen Anilkumar
Published: (2025)

AdaQAT: Adaptive Bit-Width Quantization-Aware Training
by: Gernigon, Cédric, et al.
Published: (2024)

Heart Disease Prediction: A Comparative Study of Optimisers Performance in Deep Neural Networks
by: Chibuike, Chisom, et al.
Published: (2025)

Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
by: Bushnaq, Lucius, et al.
Published: (2024)

Optimal Strategy in "Guess Who?": Beyond Binary Search
by: Nica, Mihai
Published: (2015)

Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
by: Bordelon, Blake, et al.
Published: (2025)

Confidence-based Estimators for Predictive Performance in Model Monitoring
by: Kivimäki, Juhani, et al.
Published: (2024)

Logarithmic Width Suffices for Robust Memorization
by: Egosi, Amitsour, et al.
Published: (2025)