Imagen de Portada

Stabilizing Backpropagation in 16-bit Neural Training with Modified Adam Optimizer

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Yun, Juyoung
Formato:	Preprint
Publicado:	2023
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

The Hidden Power of Pure 16-bit Floating-Point Neural Networks
por: Yun, Juyoung, et al.
Publicado: (2023)

Robust Neural Pruning with Gradient Sampling Optimization for Residual Neural Networks
por: Yun, Juyoung
Publicado: (2023)

Revisiting 16-bit Neural Network Training: A Practical Approach for Resource-Limited Learning
por: Yun, Juyoung, et al.
Publicado: (2023)

Stochastic Gradient Sampling for Enhancing Neural Networks Training
por: Yun, Juyoung
Publicado: (2023)

ZNorm: Z-Score Gradient Normalization Accelerating Skip-Connected Network Training without Architectural Modification
por: Yun, Juyoung
Publicado: (2024)

Mitigating Gradient Overlap in Deep Residual Networks with Gradient Normalization for Improved Non-Convex Optimization
por: Yun, Juyoung
Publicado: (2024)

Backpropagation Neural Tree
por: Ojha, Varun, et al.
Publicado: (2022)

Dynamic Spectral Backpropagation for Efficient Neural Network Training
por: Muthuraman, Mannmohan
Publicado: (2025)

Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation
por: Kang, Qiyu, et al.
Publicado: (2025)

Optimal Control Theoretic Neural Optimizer: From Backpropagation to Dynamic Programming
por: Liu, Guan-Horng, et al.
Publicado: (2025)

Predictive Modeling of Coronal Hole Areas Using Long Short-Term Memory Networks
por: Yun, Juyoung
Publicado: (2023)

THDC: Training Hyperdimensional Computing Models with Backpropagation
por: Dejonghe, Hanne, et al.
Publicado: (2026)

Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats
por: Chmiel, Brian, et al.
Publicado: (2021)

Empirical Results for Adjusting Truncated Backpropagation Through Time while Training Neural Audio Effects
por: Bourdin, Yann, et al.
Publicado: (2025)

Rethinking Deep Learning: Propagating Information in Neural Networks without Backpropagation and Statistical Optimization
por: Itoh, Kei
Publicado: (2024)

Fast and Interpretable Autoregressive Estimation with Neural Network Backpropagation
por: Lucena, Anaísa, et al.
Publicado: (2026)

Extreme Solar Flare Prediction Using Residual Networks with HMI Magnetograms and Intensitygrams
por: Yun, Juyoung, et al.
Publicado: (2024)

Stabilizing Backpropagation Through Time to Learn Complex Physics
por: Schnell, Patrick, et al.
Publicado: (2024)

Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation
por: Yu, Chengting, et al.
Publicado: (2024)

Advancing On-Device Neural Network Training with TinyPropv2: Dynamic, Sparse, and Efficient Backpropagation
por: Rüb, Marcus, et al.
Publicado: (2024)

Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
por: Panda, Ashwinee, et al.
Publicado: (2025)

Training Implicit Networks for Image Deblurring using Jacobian-Free Backpropagation
por: Liu, Linghai, et al.
Publicado: (2024)

Beyond Backpropagation: Exploring Innovative Algorithms for Energy-Efficient Deep Neural Network Training
por: Spyra, Przemysław
Publicado: (2025)

Subspace Optimization for Backpropagation-Free Continual Test-Time Adaptation
por: Sójka, Damian, et al.
Publicado: (2026)

AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training
por: Lu, Binghang, et al.
Publicado: (2026)

Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models?
por: Nielsen, Jacob, et al.
Publicado: (2025)

The Cost of Avoiding Backpropagation
por: Panchal, Kunjal, et al.
Publicado: (2025)

SAL: Selective Adaptive Learning for Backpropagation-Free Training with Sparsification
por: Liu, Fanping, et al.
Publicado: (2026)

Analysis and Predictive Modeling of Solar Coronal Holes Using Computer Vision and ARIMA-LSTM Networks
por: Yun, Juyoung, et al.
Publicado: (2024)

Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients
por: Flügel, Katharina, et al.
Publicado: (2024)

LPGD: A General Framework for Backpropagation through Embedded Optimization Layers
por: Paulus, Anselm, et al.
Publicado: (2024)

The Epochal Sawtooth Phenomenon: Unveiling Training Loss Oscillations in Adam and Other Optimizers
por: Liu, Qi, et al.
Publicado: (2024)

Time Series Compression using Quaternion Valued Neural Networks and Quaternion Backpropagation
por: Pöppelbaum, Johannes, et al.
Publicado: (2024)

Comprehensive Survey of Complex-Valued Neural Networks: Insights into Backpropagation and Activation Functions
por: Hammad, M. M.
Publicado: (2024)

FFCL: Forward-Forward Net with Cortical Loops, Training and Inference on Edge Without Backpropagation
por: Karkehabadi, Ali, et al.
Publicado: (2024)

ADOPT: Modified Adam Can Converge with Any $β_2$ with the Optimal Rate
por: Taniguchi, Shohei, et al.
Publicado: (2024)

SOAP: Improving and Stabilizing Shampoo using Adam
por: Vyas, Nikhil, et al.
Publicado: (2024)

Irrational Complex Rotations Empower Low-bit Optimizers
por: Tian, Zhen, et al.
Publicado: (2025)

Phasor Memory Networks: Stable Backpropagation Through Time for Scalable Explicit Memory
por: Goo, Sungwoo, et al.
Publicado: (2026)

COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization
por: Zhang, Aozhong, et al.
Publicado: (2024)