Copertina

Mixture-of-Subspaces in Low-Rank Adaptation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wu, Taiqiang, Wang, Jiahao, Zhao, Zhe, Wong, Ngai
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
di: Wu, Taiqiang, et al.
Pubblicazione: (2023)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
di: Yang, Yifan, et al.
Pubblicazione: (2024)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
di: Yang, Runming, et al.
Pubblicazione: (2024)

Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation
di: Wang, Dianyun, et al.
Pubblicazione: (2025)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
di: Tang, Chuanyu, et al.
Pubblicazione: (2024)

TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models
di: Mu, Lin, et al.
Pubblicazione: (2026)

Mixture of Low Rank Adaptation with Partial Parameter Sharing for Time Series Forecasting
di: Pan, Licheng, et al.
Pubblicazione: (2025)

MCUBERT: Memory-Efficient BERT Inference on Commodity Microcontrollers
di: Yang, Zebin, et al.
Pubblicazione: (2024)

CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models
di: Xiao, Xiaojun, et al.
Pubblicazione: (2024)

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
di: Wei, Jia, et al.
Pubblicazione: (2026)

CERSA: Cumulative Energy-Retaining Subspace Adaptation for Memory-Efficient Fine-Tuning
di: Ge, Jingze, et al.
Pubblicazione: (2026)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
di: Wu, Taiqiang, et al.
Pubblicazione: (2024)

Timber: Training-free Instruct Model Refining with Base via Effective Rank
di: Wu, Taiqiang, et al.
Pubblicazione: (2025)

The Primacy of Magnitude in Low-Rank Adaptation
di: Zhang, Zicheng, et al.
Pubblicazione: (2025)

GraphLoRA: Structure-Aware Contrastive Low-Rank Adaptation for Cross-Graph Transfer Learning
di: Yang, Zhe-Rui, et al.
Pubblicazione: (2024)

L-MoE: End-to-End Training of a Lightweight Mixture of Low-Rank Adaptation Experts
di: Ji, Shihao, et al.
Pubblicazione: (2025)

Catching a Moving Subspace: Low-Rank Bandits Beyond Stationarity
di: Khosravi, Hamed, et al.
Pubblicazione: (2026)

SRLoRA: Subspace Recomposition in Low-Rank Adaptation via Importance-Based Fusion and Reinitialization
di: Yang, Haodong, et al.
Pubblicazione: (2025)

Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
di: Samplawski, Colin, et al.
Pubblicazione: (2025)

Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
di: Wu, Yize, et al.
Pubblicazione: (2026)

A Shared Low-Rank Adaptation Approach to Personalized RLHF
di: Liu, Renpu, et al.
Pubblicazione: (2025)

D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
di: Li, Junlin, et al.
Pubblicazione: (2026)

Strategic Over-Parameterization for Generalizable Low-Rank Adaptation
di: Gao, Jing, et al.
Pubblicazione: (2026)

Few-Shot Concept Unlearning with Low Rank Adaptation
di: Shreyas, Udaya, et al.
Pubblicazione: (2025)

Energy-Structured Low-Rank Adaptation for Continual Learning
di: Li, Longhua, et al.
Pubblicazione: (2026)

Low Tensor-Rank Adaptation of Kolmogorov--Arnold Networks
di: Gao, Yihang, et al.
Pubblicazione: (2025)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
di: Sun, Mengyang, et al.
Pubblicazione: (2025)

Revisiting Model Interpolation for Efficient Reasoning
di: Wu, Taiqiang, et al.
Pubblicazione: (2025)

Lotus: Efficient LLM Training by Randomized Low-Rank Gradient Projection with Adaptive Subspace Switching
di: Miao, Tianhao, et al.
Pubblicazione: (2026)

FFT-based Dynamic Subspace Selection for Low-Rank Adaptive Optimization of Large Language Models
di: Modoranu, Ionut-Vlad, et al.
Pubblicazione: (2025)

Efficient Handwriting-Based Alzheimer,s Disease Diagnosis Using a Low-Rank Mixture of Experts Deep Learning Framework
di: Wang, Wu, et al.
Pubblicazione: (2026)

Preventing Rank Collapse in Federated Low-Rank Adaptation with Client Heterogeneity
di: Wu, Fei, et al.
Pubblicazione: (2026)

Low-Rank Compression of Pretrained Models via Randomized Subspace Iteration
di: Pourkamali-Anaraki, Farhad
Pubblicazione: (2026)

GeoRA: Geometry-Aware Low-Rank Adaptation for RLVR
di: Zhang, Jiaying, et al.
Pubblicazione: (2026)

The Expressive Power of Low-Rank Adaptation
di: Zeng, Yuchen, et al.
Pubblicazione: (2023)

SBoRA: Low-Rank Adaptation with Regional Weight Updates
di: Po, Lai-Man, et al.
Pubblicazione: (2024)

Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
di: Zhang, Wenhao, et al.
Pubblicazione: (2026)

Test-time Adaptation for Regression by Subspace Alignment
di: Adachi, Kazuki, et al.
Pubblicazione: (2024)

L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
di: Yang, Minghao, et al.
Pubblicazione: (2026)

Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
di: Zhao, Ziyu, et al.
Pubblicazione: (2025)