Copertina

Deep Neural Network Initialization with Sparsity Inducing Activations

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Price, Ilan, Ball, Nicholas Daultry, Lam, Samuel C. H., Jones, Adam C., Tanner, Jared
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Theory of Minimal Weight Perturbations in Deep Networks and its Applications for Low-Rank Activated Backdoor Attacks
di: Evans, Bethan, et al.
Pubblicazione: (2026)

On the Hardness of Training Deep Neural Networks Discretely
di: Doron-Arad, Ilan
Pubblicazione: (2024)

How Controlling the Variance can Improve Training Stability of Sparsely Activated DNNs and CNNs
di: Dent, Emily, et al.
Pubblicazione: (2026)

Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes
di: Nait-Saada, Thiziri, et al.
Pubblicazione: (2023)

SPADE: Sparsity-Guided Debugging for Deep Neural Networks
di: Moakhar, Arshia Soltani, et al.
Pubblicazione: (2023)

Chordal Sparsity for Lipschitz Constant Estimation of Deep Neural Networks
di: Xue, Anton, et al.
Pubblicazione: (2022)

Effects of Initialization Biases on Deep Neural Network Training Dynamics
di: Pellegrino, Nicholas, et al.
Pubblicazione: (2025)

Investigating Sparsity in Recurrent Neural Networks
di: Darji, Harshil
Pubblicazione: (2024)

Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks
di: Kogler, Constantin, et al.
Pubblicazione: (2026)

Why ReLU? A Bit-Model Dichotomy for Deep Network Training
di: Doron-Arad, Ilan, et al.
Pubblicazione: (2026)

Exploiting Subgradient Sparsity in Max-Plus Neural Networks
di: Enaieh, Ikhlas, et al.
Pubblicazione: (2026)

Optimal Condition for Initialization Variance in Deep Neural Networks: An SGD Dynamics Perspective
di: Horii, Hiroshi, et al.
Pubblicazione: (2025)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
di: Lee, Hyungu, et al.
Pubblicazione: (2025)

Weight Initialization and Variance Dynamics in Deep Neural Networks and Large Language Models
di: Han, Yankun
Pubblicazione: (2025)

Approximate Multiplier Induced Error Propagation in Deep Neural Networks
di: Alahakoon, A. M. H. H., et al.
Pubblicazione: (2025)

Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers
di: Saada, Thiziri Nait, et al.
Pubblicazione: (2024)

Sparsity-Aware Communication for Distributed Graph Neural Network Training
di: Mukhodopadhyay, Ujjaini, et al.
Pubblicazione: (2025)

Exploring and Improving Initialization for Deep Graph Neural Networks: A Signal Propagation Perspective
di: Wang, Senmiao, et al.
Pubblicazione: (2025)

Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations
di: Kumar, Akshay, et al.
Pubblicazione: (2024)

Sparsity-Induced Global Matrix Autoregressive Model with Auxiliary Network Data
di: Wu, Sanyou, et al.
Pubblicazione: (2025)

Chordal Sparsity for SDP-based Neural Network Verification
di: Xue, Anton, et al.
Pubblicazione: (2022)

Parallel Algorithms for Exact Enumeration of Deep Neural Network Activation Regions
di: Drammis, Sabrina, et al.
Pubblicazione: (2024)

Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning
di: Ma, Guozheng, et al.
Pubblicazione: (2025)

Universal Properties of Activation Sparsity in Modern Large Language Models
di: Szatkowski, Filip, et al.
Pubblicazione: (2025)

SAUC: Sparsity-Aware Uncertainty Calibration for Spatiotemporal Prediction with Graph Neural Networks
di: Zhuang, Dingyi, et al.
Pubblicazione: (2024)

Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient
di: Dinh, Vu C., et al.
Pubblicazione: (2024)

Activation Bottleneck: Sigmoidal Neural Networks Cannot Forecast a Straight Line
di: Toller, Maximilian, et al.
Pubblicazione: (2024)

On Unbalanced Optimal Transport: Gradient Methods, Sparsity and Approximation Error
di: Nguyen, Quang Minh, et al.
Pubblicazione: (2022)

Semiring Activation in Neural Networks
di: Smets, Bart M. N., et al.
Pubblicazione: (2024)

1-Lipschitz Neural Networks are more expressive with N-Activations
di: Prach, Bernd, et al.
Pubblicazione: (2023)

Principal Components for Neural Network Initialization
di: Phan, Nhan, et al.
Pubblicazione: (2025)

A Proximal Operator for Inducing 2:4-Sparsity
di: Kübler, Jonas M, et al.
Pubblicazione: (2025)

Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK
di: Yang, Hongru, et al.
Pubblicazione: (2023)

DQA: An Efficient Method for Deep Quantization of Deep Neural Network Activations
di: Hu, Wenhao, et al.
Pubblicazione: (2024)

Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion
di: Szatkowski, Filip, et al.
Pubblicazione: (2023)

R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference
di: Zhang, Zhenyu, et al.
Pubblicazione: (2025)

Graph Neural Networks with Coarse- and Fine-Grained Division for Mitigating Label Sparsity and Noise
di: Li, Shuangjie, et al.
Pubblicazione: (2024)

Rethinking the Relationship between Recurrent and Non-Recurrent Neural Networks: A Study in Sparsity
di: Hershey, Quincy, et al.
Pubblicazione: (2024)

Post-Training Statistical Calibration for Higher Activation Sparsity
di: Chua, Vui Seng, et al.
Pubblicazione: (2024)

Joint Training Across Multiple Activation Sparsity Regimes
di: Wang, Haotian
Pubblicazione: (2026)