OSDN: Improving Delta Rule with Provable Online Preconditioning in Linear Attention

Documenti analoghi

• MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
  di: Li, Shuhuai, et al.
  Pubblicazione: (2026)
• Beyond $\mathcal{O}(\sqrt{T})$ Regret: Decoupling Learning and Decision-making in Online Linear Programming
  di: Gao, Wenzhi, et al.
  Pubblicazione: (2025)
• Gated Delta Networks: Improving Mamba2 with Delta Rule
  di: Yang, Songlin, et al.
  Pubblicazione: (2024)
• Parallelizing Linear Transformers with the Delta Rule over Sequence Length
  di: Yang, Songlin, et al.
  Pubblicazione: (2024)
• Decoupling Learning and Decision-Making: Breaking the $\mathcal{O}(\sqrt{T})$ Barrier in Online Resource Allocation with First-Order Methods
  di: Gao, Wenzhi, et al.
  Pubblicazione: (2024)