Cover Image

Correlations Are Ruining Your Gradient Descent

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Ahmad, Nasir
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Neural and Evolutionary Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network
by: Oh, Hyunseok, et al.
Published: (2024)

Gradient Descent, Stochastic Optimization, and Other Tales
by: Lu, Jun
Published: (2022)

Density Descent for Diversity Optimization
by: Lee, David H., et al.
Published: (2023)

Mirror Descent Policy Optimisation for Robust Constrained Markov Decision Processes
by: Bossens, David M., et al.
Published: (2025)

Flexible and Efficient Surrogate Gradient Modeling with Forward Gradient Injection
by: Otte, Sebastian
Published: (2024)

Subspace Node Pruning
by: Offergeld, Joshua, et al.
Published: (2024)

Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation
by: Yang, Yibo, et al.
Published: (2024)

Gradient-based inference of abstract task representations for generalization in neural networks
by: Hummos, Ali, et al.
Published: (2024)

Gradient-Based Neuroplastic Adaptation for Concurrent Optimization of Neuro-Fuzzy Networks
by: Hostetter, John Wesley, et al.
Published: (2025)

Efficient Online Learning with Predictive Coding Networks: Exploiting Temporal Correlations
by: Zadeh-Jousdani, Darius Masoum, et al.
Published: (2025)

Conditional Finite Mixtures of Poisson Distributions for Context-Dependent Neural Correlations
by: Sokoloski, Sacha, et al.
Published: (2019)

Forward Direct Feedback Alignment for Online Gradient Estimates of Spiking Neural Networks
by: Bacho, Florian, et al.
Published: (2024)

AdaSwarm: Augmenting Gradient-Based optimizers in Deep Learning with Swarm Intelligence
by: Mohapatra, Rohan, et al.
Published: (2020)

Heterogeneous Connectivity in Sparse Networks: Fan-in Profiles, Gradient Hierarchy, and Topological Equilibria
by: Tomczak, Nikodem
Published: (2026)

On the Privacy-Preserving Properties of Spiking Neural Networks with Unique Surrogate Gradients and Quantization Levels
by: Moshruba, Ayana, et al.
Published: (2025)

SAGRAD: A Program for Neural Network Training with Simulated Annealing and the Conjugate Gradient Method
by: Bernal, Javier, et al.
Published: (2025)

SpikingGamma: Surrogate-Gradient Free and Temporally Precise Online Training of Spiking Neural Networks with Smoothed Delays
by: Koopman, Roel, et al.
Published: (2026)

Effective Predictive Modeling for Emergency Department Visits and Evaluating Exogenous Variables Impact: Using Explainable Meta-learning Gradient Boosting
by: Neshat, Mehdi, et al.
Published: (2024)

Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning
by: Descours, Arnaud, et al.
Published: (2025)

HG2P: Hippocampus-inspired High-reward Graph and Model-Free Q-Gradient Penalty for Path Planning and Motion Control
by: Wang, Haoran, et al.
Published: (2024)

Hybrid Approach to Parallel Stochastic Gradient Descent
by: Vora, Aakash Sudhirbhai, et al.
Published: (2024)

Understanding Transformer Optimization via Gradient Heterogeneity
by: Tomihari, Akiyoshi, et al.
Published: (2025)

Arc Gradient Descent: A Geometrically Motivated Gradient Descent-based Optimiser with Phase-Aware, User-Controlled Step Dynamics (proof-of-concept)
by: Verma, Nikhil, et al.
Published: (2025)

AlphaGrad: Non-Linear Gradient Normalization Optimizer
by: Sane, Soham
Published: (2025)

Gradient descent in materia through homodyne gradient extraction
by: Boon, Marcus N., et al.
Published: (2021)

It's Not a Lottery, It's a Race: Understanding How Gradient Descent Adapts the Network's Capacity to the Task
by: Pinson, Hannah
Published: (2026)

Optimization of Discrete Parameters Using the Adaptive Gradient Method and Directed Evolution
by: Beinarovich, Andrei, et al.
Published: (2024)

A Truly Sparse and General Implementation of Gradient-Based Synaptic Plasticity
by: Lohoff, Jamie, et al.
Published: (2025)

Optimal Gradient Checkpointing for Sparse and Recurrent Architectures using Off-Chip Memory
by: Bencheikh, Wadjih, et al.
Published: (2024)

Gradient-descent hardware-aware training and deployment for mixed-signal Neuromorphic processors
by: Çakal, Uğurcan, et al.
Published: (2023)

Provable Unlearning with Gradient Ascent on Two-Layer ReLU Neural Networks
by: Melamed, Odelia, et al.
Published: (2025)

Robust Lagrangian and Adversarial Policy Gradient for Robust Constrained Markov Decision Processes
by: Bossens, David M.
Published: (2023)

Learning Gaussian Multi-Index Models with Gradient Flow: Time Complexity and Directional Convergence
by: Şimşek, Berfin, et al.
Published: (2024)

Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
by: Patankar, Dhruv, et al.
Published: (2026)

Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradients
by: Zhou, Feifan, et al.
Published: (2026)

Enabling Robust In-Context Memory and Rapid Task Adaptation in Transformers with Hebbian and Gradient-Based Plasticity
by: Chaudhary, Siddharth
Published: (2025)

Gradient-Free Continual Learning in Spiking Neural Networks via Inter-Spike Interval Regularization
by: Roy, Samrendra, et al.
Published: (2026)

Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks
by: Williams, Ezekiel, et al.
Published: (2026)

Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering
by: Postmus, Joris, et al.
Published: (2024)

Efficient Estimation of Unique Components in Independent Component Analysis by Matrix Representation
by: Matsuda, Yoshitatsu, et al.
Published: (2024)