Cover Image

Understanding SOAP from the Perspective of Gradient Whitening

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lu, Yanqing, Wang, Letao, Liu, Jinbo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Understanding and Improving Shampoo and SOAP via Kullback-Leibler Minimization
by: Lin, Wu, et al.
Published: (2025)

Zipfian Whitening
by: Yokoi, Sho, et al.
Published: (2024)

Skin-SOAP: A Weakly Supervised Framework for Generating Structured SOAP Notes
by: Kamal, Sadia, et al.
Published: (2025)

Whitening Not Recommended for Classification Tasks in LLMs
by: Forooghi, Ali, et al.
Published: (2024)

SOAP: Improving and Stabilizing Shampoo using Adam
by: Vyas, Nikhil, et al.
Published: (2024)

Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage
by: Milligan, Alan, et al.
Published: (2026)

What Really Matters in Matrix-Whitening Optimizers?
by: Frans, Kevin, et al.
Published: (2025)

Whitening Spherical Gaussian Mixtures in the Large-Dimensional Regime
by: Boudjemaa, Mohammed Racim Moussa, et al.
Published: (2025)

Data Whitening Improves Sparse Autoencoder Learning
by: Saraswatula, Ashwin, et al.
Published: (2025)

A Stable Whitening Optimizer for Efficient Neural Network Training
by: Frans, Kevin, et al.
Published: (2025)

Conditionally Whitened Generative Models for Probabilistic Time Series Forecasting
by: Yang, Yanfeng, et al.
Published: (2025)

Improving Relative Representations with Learned Anchors and Whitened Inner Products
by: Svendsen, Oscar Thorsted, et al.
Published: (2026)

SOAP-RL: Sequential Option Advantage Propagation for Reinforcement Learning in POMDP Environments
by: Ishida, Shu, et al.
Published: (2024)

Exploring Robustness in Doctor-Patient Conversation Summarization: An Analysis of Out-of-Domain SOAP Notes
by: Chen, Yu-Wen, et al.
Published: (2024)

Training-Free Policy Violation Detection via Activation-Space Whitening in LLMs
by: Rachmil, Oren, et al.
Published: (2025)

Out-of-Distribution Radar Detection with Complex VAEs: Theory, Whitening, and ANMF Fusion
by: Rouzoumka, Yadang Alexis, et al.
Published: (2026)

IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression
by: Abbasi, Ali, et al.
Published: (2026)

SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training
by: Ma, Chao, et al.
Published: (2024)

Pro-KLShampoo: Projected KL-Shampoo with Whitening Recovered by Orthogonalization
by: Sun, Ruotong, et al.
Published: (2026)

An Integrated Fusion Framework for Ensemble Learning Leveraging Gradient Boosting and Fuzzy Rule-Based Models
by: Li, Jinbo, et al.
Published: (2025)

A Bootstrap Perspective on Stochastic Gradient Descent
by: Lan, Hongjian, et al.
Published: (2025)

Controllable Feature Whitening for Hyperparameter-Free Bias Mitigation
by: Cho, Yooshin, et al.
Published: (2025)

Reservoir Subspace Injection for Online ICA under Top-n Whitening
by: Xiao, Wenjun, et al.
Published: (2026)

Gradient Guidance for Diffusion Models: An Optimization Perspective
by: Guo, Yingqing, et al.
Published: (2024)

Towards Scalable SOAP Note Generation: A Weakly Supervised Multimodal Framework
by: Kamal, Sadia, et al.
Published: (2025)

Understanding Robust Overfitting from the Feature Generalization Perspective
by: Yu, Chaojian, et al.
Published: (2023)

When Reasoning Hurts: Source-Aware Evaluation of Frontier LLMs for Clinical SOAP Note Generation
by: Faisal, Faizan
Published: (2026)

GradPower: Powering Gradients for Faster Language Model Pre-Training
by: Wang, Jinbo, et al.
Published: (2025)

Principled Understanding of Generalization for Generative Transformer Models in Arithmetic Reasoning Tasks
by: Xu, Xingcheng, et al.
Published: (2024)

Understanding Private Learning From Feature Perspective
by: Ding, Meng, et al.
Published: (2025)

Grouped Satisficing Paths in Pure Strategy Games: a Topological Perspective
by: Fu, Yanqing, et al.
Published: (2025)

It Ain't That Bad: Understanding the Mysterious Performance Drop in OOD Generalization for Generative Transformer Models
by: Xu, Xingcheng, et al.
Published: (2023)

Gradient Shaping Beyond Clipping: A Functional Perspective on Update Magnitude Control
by: You, Haochen, et al.
Published: (2025)

Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective
by: Qin, Shaoxiang, et al.
Published: (2024)

Understanding Stochastic Natural Gradient Variational Inference
by: Wu, Kaiwen, et al.
Published: (2024)

Understanding Gradient Descent through the Training Jacobian
by: Belrose, Nora, et al.
Published: (2024)

Rethinking the Diffusion Models for Numerical Tabular Data Imputation from the Perspective of Wasserstein Gradient Flow
by: Chen, Zhichao, et al.
Published: (2024)

A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning
by: Feng, Xidong, et al.
Published: (2021)

Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond
by: Wang, Qizhou, et al.
Published: (2025)

Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulation
by: Gu, Shangding, et al.
Published: (2024)