Image de couverture de livre

Reconstructing Deep Neural Networks: Unleashing the Optimization Potential of Natural Gradient Descent

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liu, Weihua, Boumaraf, Said, Li, Jianwu, Lin, Chaochao, Liu, Xiabi, Niu, Lijuan, Werghi, Naoufel
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

BENet: A Cross-domain Robust Network for Detecting Face Forgeries via Bias Expansion and Latent-space Attention
par: Liu, Weihua, et autres
Publié: (2024)

Occlusion-Aware Deep Convolutional Neural Network via Homogeneous Tanh-transforms for Face Parsing
par: Qiua, Jianhua, et autres
Publié: (2023)

Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Training
par: Ma, Yuhan, et autres
Publié: (2024)

Revisiting the Initial Steps in Adaptive Gradient Descent Optimization
par: Abuduweili, Abulikemu, et autres
Publié: (2024)

ONG: Orthogonal Natural Gradient Descent
par: Yadav, Yajat, et autres
Publié: (2025)

PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization
par: Xu, Mingjing, et autres
Publié: (2024)

Conflict-Averse Gradient Descent for Multi-task Learning
par: Liu, Bo, et autres
Publié: (2021)

Optimization, Generalization and Differential Privacy Bounds for Gradient Descent on Kolmogorov-Arnold Networks
par: Wang, Puyu, et autres
Publié: (2026)

Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning
par: Garg, Ishir, et autres
Publié: (2026)

Deep Distance Map Regression Network with Shape-aware Loss for Imbalanced Medical Image Segmentation
par: Li, Huiyu, et autres
Publié: (2025)

Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent
par: Li, Pingzhi, et autres
Publié: (2024)

Multi-Modal Attention Networks for Enhanced Segmentation and Depth Estimation of Subsurface Defects in Pulse Thermography
par: Salah, Mohammed, et autres
Publié: (2025)

Trustworthiness of Stochastic Gradient Descent in Distributed Learning
par: Li, Hongyang, et autres
Publié: (2024)

Liquid Neural Network Models for Natural Gas Spot Price Time-Series Forecasting
par: Liu, Yiqian, et autres
Publié: (2026)

Noise Balance and Stationary Distribution of Stochastic Gradient Descent
par: Ziyin, Liu, et autres
Publié: (2023)

Cross-Entropy Optimization for Hyperparameter Optimization in Stochastic Gradient-based Approaches to Train Deep Neural Networks
par: Li, Kevin, et autres
Publié: (2024)

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
par: Cheng, Wenhua, et autres
Publié: (2023)

Natural Gradient Descent for Online Continual Learning
par: Khawand, Joe, et autres
Publié: (2026)

SoD$^2$: Statically Optimizing Dynamic Deep Neural Network
par: Niu, Wei, et autres
Publié: (2024)

DeepDefense: Layer-Wise Gradient-Feature Alignment for Building Robust Neural Networks
par: Lin, Ci, et autres
Publié: (2025)

Heterogeneous Temporal Hypergraph Neural Network
par: Liu, Huan, et autres
Publié: (2025)

Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization
par: Kumar, Ramnath, et autres
Publié: (2023)

Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training
par: Lu, Yishun, et autres
Publié: (2025)

Gradient Descent Algorithm Survey
par: Fucheng, Deng, et autres
Publié: (2025)

FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning
par: Liu, Junkang, et autres
Publié: (2026)

Graph Attention Networks Unleashed: A Fast and Explainable Vulnerability Assessment Framework for Microgrids
par: Liu, Wei, et autres
Publié: (2025)

Gradient Inversion Attack on Graph Neural Networks
par: Sinha, Divya Anand, et autres
Publié: (2024)

Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving
par: Zheng, Yinan, et autres
Publié: (2026)

SURGE: Surrogate Gradient Adaptation in Binary Neural Networks
par: Huang, Haoyu, et autres
Publié: (2026)

Randomness and Interpolation Improve Gradient Descent
par: Li, Jiawen, et autres
Publié: (2025)

Learning Associative Memories with Gradient Descent
par: Cabannes, Vivien, et autres
Publié: (2024)

On the Convergence of (Stochastic) Gradient Descent for Kolmogorov--Arnold Networks
par: Gao, Yihang, et autres
Publié: (2024)

Mitigating Gradient Overlap in Deep Residual Networks with Gradient Normalization for Improved Non-Convex Optimization
par: Yun, Juyoung
Publié: (2024)

Auto-Unrolled Proximal Gradient Descent: An AutoML Approach to Interpretable Waveform Optimization
par: Kaplan, Ahmet
Publié: (2026)

Optimizing Predictive AI in Physical Design Flows with Mini Pixel Batch Gradient Descent
par: Yang, Haoyu, et autres
Publié: (2024)

Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning
par: Yang, Shan, et autres
Publié: (2026)

Neural Network Optimal Power Flow via Energy Gradient Flow and Unified Dynamics
par: Liu, Xuezhi
Publié: (2025)

Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization
par: Liang, Chen, et autres
Publié: (2026)

Vanilla Gradient Descent for Oblique Decision Trees
par: Panda, Subrat Prasad, et autres
Publié: (2024)

Adaptive Heavy-Tailed Stochastic Gradient Descent
par: Gong, Bodu, et autres
Publié: (2025)