Cover Image

$λ$-GELU: Learning Gating Hardness for Controlled ReLU-ization in Deep Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pérez-Corral, Cristian, Fernández-Hernández, Alberto, Mestre, Jose I., Dolz, Manuel F., Quintana-Ortí, Enrique S.
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

StableGrad: Backward Scale Control without Batch Normalization
by: Mestre, Jose I., et al.
Published: (2026)

FedSQ: Optimized Weight Averaging via Fixed Gating
by: Pérez-Corral, Cristian, et al.
Published: (2026)

Refresh-Scaling the Memory of Balanced Adam
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

Why Adam Works Better with $β_1 = β_2$: The Missing Gradient Scale Invariance Principle
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training
by: Pérez-Corral, Cristian, et al.
Published: (2026)

OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

OUIDecay: Adaptive Layer-wise Weight Decay for CNNs Using Online Activation Patterns
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

FedOUI: OUI-Guided Client Weighting for Federated Aggregation
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

When Learning Rates Go Wrong: Early Structural Signals in PPO Actor-Critic
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

GLAI: GreenLightningAI for Accelerated Training through Knowledge Decoupling
by: Mestre, Jose I., et al.
Published: (2025)

Detecting Atypical Clients in Federated Learning via Representation-Level Divergence
by: Pérez-Corral, Cristian, et al.
Published: (2026)

OUI Need to Talk About Weight Decay: A New Perspective on Overfitting Detection
by: Fernández-Hernández, Alberto, et al.
Published: (2025)

Sinusoidal Initialization, Time for a New Start
by: Fernández-Hernández, Alberto, et al.
Published: (2025)

Zorro: A Flexible and Differentiable Parametric Family of Activation Functions That Extends ReLU and GELU
by: Roodschild, Matias, et al.
Published: (2024)

Constructive Universal Approximation and Finite Sample Memorization by Narrow Deep ReLU Networks
by: Hernández, Martín, et al.
Published: (2024)

The Geometry of ReLU Networks through the ReLU Transition Graph
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
by: Dhayalkar, Sahil Rajesh
Published: (2025)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
by: Manik, Md Motaleb Hossen, et al.
Published: (2025)

SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks
by: Sun, Xiaotong, et al.
Published: (2024)

The Resurrection of the ReLU
by: Horuz, Coşku Can, et al.
Published: (2025)

On the Local Complexity of Linear Regions in Deep ReLU Networks
by: Patel, Niket, et al.
Published: (2024)

Deep ReLU Networks Have Surprisingly Simple Polytopes
by: Fan, Feng-Lei, et al.
Published: (2023)

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks
by: Vallin, Jonatan, et al.
Published: (2024)

Geometry-induced Regularization in Deep ReLU Neural Networks
by: Bona-Pellissier, Joachim, et al.
Published: (2024)

Competition-based Adaptive ReLU for Deep Neural Networks
by: Chen, Junjia, et al.
Published: (2024)

The Symmetries of Three-Layer ReLU Networks
by: Gegenfurtner, Johanna Marie, et al.
Published: (2026)

Topological Expressivity of ReLU Neural Networks
by: Ergen, Ekin, et al.
Published: (2023)

Stochastic Bandits with ReLU Neural Networks
by: Xu, Kan, et al.
Published: (2024)

On Space Folds of ReLU Neural Networks
by: Lewandowski, Michal, et al.
Published: (2025)

Three Quantization Regimes for ReLU Networks
by: Ou, Weigutian, et al.
Published: (2024)

Pathwise Explanation of ReLU Neural Networks
by: Lim, Seongwoo, et al.
Published: (2025)

Robustness of Deep ReLU Networks to Misclassification of High-Dimensional Data
by: Kůrková, Věra
Published: (2026)

Beyond ReLU: Chebyshev-DQN for Enhanced Deep Q-Networks
by: Yazdannik, Saman, et al.
Published: (2025)

Deep Network Approximation: Beyond ReLU to Diverse Activation Functions
by: Zhang, Shijun, et al.
Published: (2023)

Complexity of Linear Regions in Self-supervised Deep ReLU Networks
by: Muthivhi, Mufhumudzi, et al.
Published: (2026)

Sobolev Approximation of Deep ReLU Networks in Log-Barron Space
by: Song, Changhoon, et al.
Published: (2026)

Is ReLU Adversarially Robust?
by: Sooksatra, Korn, et al.
Published: (2024)

ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU
by: Qiu, Qi, et al.
Published: (2024)

Component-based Sketching for Deep ReLU Nets
by: Wang, Di, et al.
Published: (2024)

ReLU Networks as Random Functions: Their Distribution in Probability Space
by: Chaudhari, Shreyas, et al.
Published: (2025)