Cover Image

Decoupled Orthogonal Dynamics: Regularization for Deep Network Optimizers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Hao, Yuan, Jinghui, Zhang, Hanmin
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Nora: Normalized Orthogonal Row Alignment for Scalable Matrix Optimizer
by: Yuan, Jinghui, et al.
Published: (2026)

Linear Preference Optimization: Decoupled Gradient Control via Absolute Regularization
by: Wang, Rui, et al.
Published: (2025)

Slicing Input Features to Accelerate Deep Learning: A Case Study with Graph Neural Networks
by: Xu, Zhengjia, et al.
Published: (2024)

Local LMO: Constrained Gradient Optimization via a Local Linear Minimization Oracle
by: Richtárik, Peter, et al.
Published: (2026)

A Margin-Maximizing Fine-Grained Ensemble Method
by: Yuan, Jinghui, et al.
Published: (2024)

ROOT: Robust Orthogonalized Optimizer for Neural Network Training
by: He, Wei, et al.
Published: (2025)

$\boldsymbolλ$-Orthogonality Regularization for Compatible Representation Learning
by: Ricci, Simone, et al.
Published: (2025)

Riemannian Optimization on Relaxed Indicator Matrix Manifold
by: Yuan, Jinghui, et al.
Published: (2025)

Clustering-Based Weight Orthogonalization for Stabilizing Deep Reinforcement Learning
by: Ma, Guoqing, et al.
Published: (2025)

Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation
by: Liu, Hao, et al.
Published: (2024)

Orthogonal Transforms in Neural Networks Amount to Effective Regularization
by: Zając, Krzysztof, et al.
Published: (2023)

Deep Sturm--Liouville: From Sample-Based to 1D Regularization with Learnable Orthogonal Basis Functions
by: Vigouroux, David, et al.
Published: (2025)

Estimating Heterogeneous Causal Effect on Networks via Orthogonal Learning
by: Wu, Yuanchen, et al.
Published: (2025)

Adaptive Gradient Regularization: A Faster and Generalizable Optimization Technique for Deep Neural Networks
by: Jiang, Huixiu, et al.
Published: (2024)

Regularized Langevin Dynamics for Combinatorial Optimization
by: Feng, Shengyu, et al.
Published: (2025)

Forget by Uncertainty: Orthogonal Entropy Unlearning for Quantized Neural Networks
by: Zhang, Tian, et al.
Published: (2026)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
by: Zhou, Dongruo, et al.
Published: (2018)

Deep Network Trainability via Persistent Subspace Orthogonality
by: Massucco, Alex, et al.
Published: (2025)

REG: A Regularization Optimizer for Robust Training Dynamics
by: Liu, Zehua, et al.
Published: (2025)

Decoupling and Damping: Structurally-Regularized Gradient Matching for Multimodal Graph Condensation
by: Shen, Lian, et al.
Published: (2025)

Deep Modularity Networks with Diversity-Preserving Regularization
by: Salehi, Yasmin, et al.
Published: (2025)

FISMO: Fisher-Structured Momentum-Orthogonalized Optimizer
by: Xu, Chenrui, et al.
Published: (2026)

Achieving More with Less: A Tensor-Optimization-Powered Ensemble Method
by: Yuan, Jinghui, et al.
Published: (2024)

Robustly Optimized Deep Feature Decoupling Network for Fatty Liver Diseases Detection
by: Huang, Peng, et al.
Published: (2024)

Decouple and Orthogonalize: A Data-Free Framework for LoRA Merging
by: Zheng, Shenghe, et al.
Published: (2025)

Lattice-based Deep Neural Networks: Regularity and Tailored Regularization
by: Keller, Alexander, et al.
Published: (2026)

Orthogonalized Policy Optimization:Policy Optimization as Orthogonal Projection in Hilbert Space
by: Zixian, Wang
Published: (2026)

Non-Convex Optimization with Spectral Radius Regularization
by: Sandler, Adam, et al.
Published: (2021)

Quantization-Aware Regularizers for Deep Neural Networks Compression
by: Malchiodi, Dario, et al.
Published: (2026)

Adaptive Tabu Dropout for Regularization of Deep Neural Network
by: Hasan, Md. Tarek, et al.
Published: (2024)

AdaCubic: An Adaptive Cubic Regularization Optimizer for Deep Learning
by: Tsingalis, Ioannis, et al.
Published: (2026)

Improving Deep Learning Optimization through Constrained Parameter Regularization
by: Franke, Jörg K. H., et al.
Published: (2023)

Deep Orthogonal Hypersphere Compression for Anomaly Detection
by: Zhang, Yunhe, et al.
Published: (2023)

VAEs and GANs: Implicitly Approximating Complex Distributions with Simple Base Distributions and Deep Neural Networks -- Principles, Necessity, and Limitations
by: Wei, Yuan-Hao
Published: (2025)

Group Orthogonalized Policy Optimization:Group Policy Optimization as Orthogonal Projection in Hilbert Space
by: Zixian, Wang
Published: (2026)

Memory-Guided Trust-Region Bayesian Optimization (MG-TuRBO) for High Dimensions
by: Saroj, Abhilasha, et al.
Published: (2026)

Accelerating Zeroth-Order Spectral Optimization with Partial Orthogonalization from Power Iteration
by: Chen, Jiahe, et al.
Published: (2026)

A Decoupled Basis-Vector-Driven Generative Framework for Dynamic Multi-Objective Optimization
by: Yang, Yaoming, et al.
Published: (2026)

Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization
by: Kovalev, Dmitry
Published: (2025)

Identifying Intervenable and Interpretable Features via Orthogonality Regularization
by: Miller, Moritz, et al.
Published: (2026)