A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models

Documenti analoghi

• RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
  di: Sun, Mengyang, et al.
  Pubblicazione: (2026)
• Parameter-Efficient Fine-Tuning for Foundation Models
  di: Zhang, Dan, et al.
  Pubblicazione: (2025)
• SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
  di: Tang, Anke, et al.
  Pubblicazione: (2024)
• Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
  di: Cong, Peizhuang, et al.
  Pubblicazione: (2025)
• HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
  di: Wei, Jia, et al.
  Pubblicazione: (2026)