Cover Image

FISMO: Fisher-Structured Momentum-Orthogonalized Optimizer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Chenrui, Yan, Wenjing, Zhang, Ying-Jun Angela
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Problem-Parameter-Free Decentralized Bilevel Optimization
by: Zhai, Zhiwei, et al.
Published: (2025)

Feature Resemblance: Towards a Theoretical Understanding of Analogical Reasoning in Transformers
by: Xu, Ruichen, et al.
Published: (2026)

DP-Muon: Differentially Private Optimization via Matrix-Orthogonalized Momentum
by: Kim, Jihwan, et al.
Published: (2026)

On the Spectral Structure and Objective Equivalence of Orthogonal Multilabel Fisher Discriminants
by: Keith-Norambuena, Brian, et al.
Published: (2026)

TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
by: Cheng, Peng, et al.
Published: (2026)

AuON: A Linear-time Alternative to Orthogonal Momentum Updates
by: Maity, Dipan
Published: (2025)

Structured Difference-of-Q via Orthogonal Learning
by: Cao, Defu, et al.
Published: (2024)

Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning
by: Garg, Ishir, et al.
Published: (2026)

AdaFisher: Adaptive Second Order Optimization via Fisher Information
by: Gomes, Damien Martins, et al.
Published: (2024)

Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training
by: Lu, Yishun, et al.
Published: (2025)

Decentralized Multi-Task Online Convex Optimization Under Random Link Failures
by: Yan, Wenjing, et al.
Published: (2024)

Orthogonalized Policy Optimization:Policy Optimization as Orthogonal Projection in Hilbert Space
by: Zixian, Wang
Published: (2026)

Decoupled Orthogonal Dynamics: Regularization for Deep Network Optimizers
by: Chen, Hao, et al.
Published: (2026)

Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow
by: Ren, Yinuo, et al.
Published: (2023)

Dynamic Momentum Recalibration in Online Gradient Learning
by: Yao, Zhipeng, et al.
Published: (2026)

Group Orthogonalized Policy Optimization:Group Policy Optimization as Orthogonal Projection in Hilbert Space
by: Zixian, Wang
Published: (2026)

AMO: Adaptive Muon Orthogonalization
by: Zhuang, Xinlin, et al.
Published: (2026)

Structural Priors and Modular Adapters in the Composable Fine-Tuning Algorithm of Large-Scale Models
by: Wang, Yuxiao, et al.
Published: (2025)

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
by: Xie, Xingyu, et al.
Published: (2022)

Bridging Training and Merging Through Momentum-Aware Optimization
by: Moayedikia, Alireza, et al.
Published: (2025)

Random Scaling and Momentum for Non-smooth Non-convex Optimization
by: Zhang, Qinzi, et al.
Published: (2024)

Transition Flow Matching
by: Ma, Chenrui
Published: (2026)

Stochastic Difference-of-Convex Optimization with Momentum
by: Chayti, El Mahdi, et al.
Published: (2025)

DeMo: Decoupled Momentum Optimization
by: Peng, Bowen, et al.
Published: (2024)

On the Limits of Momentum in Decentralized and Federated Optimization
by: Zaccone, Riccardo, et al.
Published: (2025)

Feature Selection Based on Orthogonal Constraints and Polygon Area
by: Zhang, Zhenxing, et al.
Published: (2024)

FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning
by: Yan, Peishen, et al.
Published: (2026)

Communication-Learning Co-Design for Differentially Private Over-the-Air Federated Distillation
by: Hu, Zihao, et al.
Published: (2025)

ORTHOBO: Orthogonal Bayesian Hyperparameter Optimization
by: Schröder, Maresa, et al.
Published: (2026)

Outer-Momentum Restarting in High-Dimensional Two-Phase Optimization
by: Topollai, Kristi, et al.
Published: (2026)

Optimizing the Adversarial Perturbation with a Momentum-based Adaptive Matrix
by: Tao, Wei, et al.
Published: (2025)

Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization
by: Liu, Wei, et al.
Published: (2025)

On the Performance Analysis of Momentum Method: A Frequency Domain Perspective
by: Li, Xianliang, et al.
Published: (2024)

Orthogonal Self-Attention
by: Zhang, Leo, et al.
Published: (2026)

Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
by: Gong, Wenbo, et al.
Published: (2025)

Graph-enhanced Optimizers for Structure-aware Recommendation Embedding Evolution
by: Xu, Cong, et al.
Published: (2023)

Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Test
by: Li, Ziyue, et al.
Published: (2025)

AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training
by: Lu, Binghang, et al.
Published: (2026)

OMPQ: Orthogonal Mixed Precision Quantization
by: Ma, Yuexiao, et al.
Published: (2021)

Shuffling Momentum Gradient Algorithm for Convex Optimization
by: Tran, Trang H., et al.
Published: (2024)