Copertina

Noise-Adaptive Layerwise Learning Rates: Accelerating Geometry-Aware Optimization for Deep Neural Network Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Hao, Jie, Gong, Xiaochuan, Xu, Jie, Wang, Zhengdao, Liu, Mingrui
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization
di: Gong, Xiaochuan, et al.
Pubblicazione: (2025)

An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
di: Gong, Xiaochuan, et al.
Pubblicazione: (2024)

On the Convergence of Adam-Type Algorithm for Bilevel Optimization under Unbounded Smoothness
di: Gong, Xiaochuan, et al.
Pubblicazione: (2025)

Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
di: Hao, Jie, et al.
Pubblicazione: (2024)

A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
di: Gong, Xiaochuan, et al.
Pubblicazione: (2024)

Bilevel Optimization with Lower-Level Uniform Convexity: Theory and Algorithm
di: Wu, Yuman, et al.
Pubblicazione: (2026)

Layerwise LQR for Geometry-Aware Optimization of Deep Networks
di: Dufort-Labbé, Simon, et al.
Pubblicazione: (2026)

An Adaptive and Stability-Promoting Layerwise Training Approach for Sparse Deep Neural Network Architecture
di: Krishnanunni, C G, et al.
Pubblicazione: (2022)

Layerwise Progressive Freezing Enables STE-Free Training of Deep Binary Neural Networks
di: Smith, Evan Gibson, et al.
Pubblicazione: (2026)

Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space
di: Chen, Zhengdao
Pubblicazione: (2023)

Semantic-Aware Gaussian Process Calibration with Structured Layerwise Kernels for Deep Neural Networks
di: Lee, Kyung-hwan, et al.
Pubblicazione: (2025)

BALI: Learning Neural Networks via Bayesian Layerwise Inference
di: Kurle, Richard, et al.
Pubblicazione: (2024)

Layerwise Change of Knowledge in Neural Networks
di: Cheng, Xu, et al.
Pubblicazione: (2024)

Efficient PAC Learning of Halfspaces with Constant Malicious Noise Rate
di: Shen, Jie
Pubblicazione: (2024)

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining
di: Hao, Jie, et al.
Pubblicazione: (2025)

Geometry-Aware Neural Optimizer for Shape Optimization and Inversion
di: Sun, Guoze, et al.
Pubblicazione: (2026)

Deep Symbolic Optimization for Combinatorial Optimization: Accelerating Node Selection by Discovering Potential Heuristics
di: Liu, Hongyu, et al.
Pubblicazione: (2024)

AdaptGrad: Adaptive Sampling to Reduce Noise
di: Zhou, Linjiang, et al.
Pubblicazione: (2024)

Attribute-Efficient PAC Learning of Sparse Halfspaces with Constant Malicious Noise Rate
di: Zeng, Shiwei, et al.
Pubblicazione: (2025)

Learning Rate Optimization for Deep Neural Networks Using Lipschitz Bandits
di: Priyanka, Padma, et al.
Pubblicazione: (2024)

Deep Neural Network Training as Random Effects: An Optimization-Inference Duality
di: Yao, Minhao, et al.
Pubblicazione: (2026)

Beyond Single-Model Views for Deep Learning: Optimization versus Generalizability of Stochastic Optimization Algorithms
di: Inan, Toki Tahmid, et al.
Pubblicazione: (2024)

One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs
di: He, Di, et al.
Pubblicazione: (2026)

Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks
di: Zou, Jinping, et al.
Pubblicazione: (2024)

Deep Neural Network Solutions for Oscillatory Fredholm Integral Equations
di: Jiang, Jie, et al.
Pubblicazione: (2024)

On the Interpolation Effect of Score Smoothing in Diffusion Models
di: Chen, Zhengdao
Pubblicazione: (2025)

Layerwise Recall and the Geometry of Interwoven Knowledge in LLMs
di: Lei, Ge, et al.
Pubblicazione: (2025)

A Triple-Inertial Accelerated Alternating Optimization Method for Deep Learning Training
di: Yan, Chengcheng, et al.
Pubblicazione: (2025)

Adaptive Layerwise Perturbation: Unifying Off-Policy Corrections for LLM RL
di: Ye, Chenlu, et al.
Pubblicazione: (2026)

Training on the Edge of Stability Is Caused by Layerwise Jacobian Alignment
di: Lowell, Mark, et al.
Pubblicazione: (2024)

Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise
di: Zhang, Jiayu, et al.
Pubblicazione: (2026)

Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation
di: Yu, Chengting, et al.
Pubblicazione: (2024)

Robust Deep Reinforcement Learning with Adaptive Adversarial Perturbations in Action Space
di: Liu, Qianmei, et al.
Pubblicazione: (2024)

Geometry Aware Meta-Learning Neural Network for Joint Phase and Precoder Optimization in RIS
di: Devapriya, Dahlia, et al.
Pubblicazione: (2024)

Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation
di: Liu, Hao, et al.
Pubblicazione: (2024)

μP$^2$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
di: Haas, Moritz, et al.
Pubblicazione: (2024)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
di: Crawshaw, Michael, et al.
Pubblicazione: (2025)

MoNTA: Accelerating Mixture-of-Experts Training with Network-Traffc-Aware Parallel Optimization
di: Guo, Jingming, et al.
Pubblicazione: (2024)

Personalized Federated Fine-tuning for Heterogeneous Data: An Automatic Rank Learning Approach via Two-Level LoRA
di: Hao, Jie, et al.
Pubblicazione: (2025)

Complexity-Aware Training of Deep Neural Networks for Optimal Structure Discovery
di: Guenter, Valentin Frank Ingmar, et al.
Pubblicazione: (2024)