Copertina

Taming Momentum: Rethinking Optimizer States Through Low-Rank Approximation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Zhengbo, Liang, Jian, He, Ran, Wang, Zilei, Tan, Tieniu
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
di: Wang, Zhengbo, et al.
Pubblicazione: (2024)

Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models
di: Wang, Zhengbo, et al.
Pubblicazione: (2024)

A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation
di: Wang, Zhengbo, et al.
Pubblicazione: (2024)

Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory
di: Liu, Yexiang, et al.
Pubblicazione: (2025)

Towards Compatible Fine-tuning for Vision-Language Model Updates
di: Wang, Zhengbo, et al.
Pubblicazione: (2024)

A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
di: Liang, Jian, et al.
Pubblicazione: (2023)

Realistic Unsupervised CLIP Fine-tuning with Universal Entropy Optimization
di: Liang, Jian, et al.
Pubblicazione: (2023)

A3 : an Analytical Low-Rank Approximation Framework for Attention
di: Wong, Jeffrey T. H., et al.
Pubblicazione: (2025)

Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs
di: Yan, Dong, et al.
Pubblicazione: (2026)

The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models
di: Sheng, Lijun, et al.
Pubblicazione: (2025)

Localized LoRA: A Structured Low-Rank Approximation for Efficient Fine-Tuning
di: Barazandeh, Babak, et al.
Pubblicazione: (2025)

CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models
di: Xiao, Xiaojun, et al.
Pubblicazione: (2024)

What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time
di: Yan, Dong, et al.
Pubblicazione: (2026)

SwitchLoRA: Switched Low-Rank Adaptation Can Learn Full-Rank Information
di: Zhou, Kaiye, et al.
Pubblicazione: (2024)

Momentum Streams for Optimizer-Inspired Transformers
di: Gai, Jingchu, et al.
Pubblicazione: (2026)

AdaPreLoRA: Adafactor Preconditioned Low-Rank Adaptation
di: Liu, Ziyun, et al.
Pubblicazione: (2026)

Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning
di: Yu, Yongcan, et al.
Pubblicazione: (2026)

Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models
di: Wu, Junfei, et al.
Pubblicazione: (2024)

Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
di: Ponkshe, Kaustubh, et al.
Pubblicazione: (2024)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
di: Shi, Haizhou, et al.
Pubblicazione: (2024)

Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation
di: Wang, Dianyun, et al.
Pubblicazione: (2025)

BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models
di: Wang, Yibin, et al.
Pubblicazione: (2024)

MemMamba: Rethinking Memory Patterns in State Space Model
di: Wang, Youjin, et al.
Pubblicazione: (2025)

Adversarial Tokenization
di: Geh, Renato Lui, et al.
Pubblicazione: (2025)

GoRA: Gradient-driven Adaptive Low Rank Adaptation
di: He, Haonan, et al.
Pubblicazione: (2025)

Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters
di: Wang, Xinjue, et al.
Pubblicazione: (2026)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
di: Tang, Chuanyu, et al.
Pubblicazione: (2024)

Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization
di: Wang, Dongwei, et al.
Pubblicazione: (2024)

SBoRA: Low-Rank Adaptation with Regional Weight Updates
di: Po, Lai-Man, et al.
Pubblicazione: (2024)

Rethinking Adapter Placement: A Dominant Adaptation Module Perspective
di: Zhang, Suoxin, et al.
Pubblicazione: (2026)

Ensembles of Low-Rank Expert Adapters
di: Li, Yinghao, et al.
Pubblicazione: (2025)

The Expressive Power of Low-Rank Adaptation
di: Zeng, Yuchen, et al.
Pubblicazione: (2023)

GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection
di: Liao, Xutao, et al.
Pubblicazione: (2024)

Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting
di: Wang, Cheng, et al.
Pubblicazione: (2026)

Exploring Vacant Classes in Label-Skewed Federated Learning
di: Guo, Kuangpu, et al.
Pubblicazione: (2024)

Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
di: Wei, Boyi, et al.
Pubblicazione: (2024)

SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
di: Zeng, Weihao, et al.
Pubblicazione: (2025)

Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization
di: Kim, Sunghwan, et al.
Pubblicazione: (2025)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
di: Yang, Yaming, et al.
Pubblicazione: (2024)

Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation
di: Li, Jin, et al.
Pubblicazione: (2025)