Copertina

LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Sun, Weigao, Lan, Disen, Zhong, Yiran, Qu, Xiaoye, Cheng, Yu
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
di: Sun, Weigao, et al.
Pubblicazione: (2025)

MoM: Linear Sequence Modeling with Mixture-of-Memories
di: Du, Jusen, et al.
Pubblicazione: (2025)

Liger: Linearizing Large Language Models to Gated Recurrent Structures
di: Lan, Disen, et al.
Pubblicazione: (2025)

Linear Attention Sequence Parallelism
di: Sun, Weigao, et al.
Pubblicazione: (2024)

Native Hybrid Attention for Efficient Sequence Modeling
di: Du, Jusen, et al.
Pubblicazione: (2025)

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
di: Qin, Zhen, et al.
Pubblicazione: (2024)

LASP: Surveying the State-of-the-Art in Large Language Model-Assisted AI Planning
di: Li, Haoming, et al.
Pubblicazione: (2024)

PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
di: Song, Mingyang, et al.
Pubblicazione: (2025)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
di: Lu, Zhenyi, et al.
Pubblicazione: (2024)

Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
di: Dentamaro, Vincenzo
Pubblicazione: (2025)

Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods
di: Benfeghoul, Martin, et al.
Pubblicazione: (2025)

On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion
di: Fan, Chenghao, et al.
Pubblicazione: (2024)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
di: MiniCPM Team, et al.
Pubblicazione: (2026)

Learning to Reason under Off-Policy Guidance
di: Yan, Jianhao, et al.
Pubblicazione: (2025)

ExGRPO: Learning to Reason from Experience
di: Zhan, Runzhe, et al.
Pubblicazione: (2025)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
di: Chen, Yingfa, et al.
Pubblicazione: (2026)

Higher-order Linear Attention
di: Zhang, Yifan, et al.
Pubblicazione: (2025)

Block-Attention for Efficient Prefilling
di: Ma, Dongyang, et al.
Pubblicazione: (2024)

Trainable Dynamic Mask Sparse Attention
di: Shi, Jingze, et al.
Pubblicazione: (2025)

Rethinking Entropy Regularization in Large Reasoning Models
di: Jiang, Yuxian, et al.
Pubblicazione: (2025)

Comba: Improving Bilinear RNNs with Closed-loop Control
di: Hu, Jiaxi, et al.
Pubblicazione: (2025)

Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow
di: Zhong, Yangyang, et al.
Pubblicazione: (2026)

Star Attention: Efficient LLM Inference over Long Sequences
di: Acharya, Shantanu, et al.
Pubblicazione: (2024)

Attention Mechanisms Don't Learn Additive Models: Rethinking Feature Importance for Transformers
di: Leemann, Tobias, et al.
Pubblicazione: (2024)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
di: Ling Team, et al.
Pubblicazione: (2025)

Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models
di: Lan, Michael, et al.
Pubblicazione: (2023)

Parallax: Parameterized Local Linear Attention for Language Modeling
di: Zuo, Yifei, et al.
Pubblicazione: (2026)

PMKLC: Parallel Multi-Knowledge Learning-based Lossless Compression for Large-Scale Genomics Database
di: Sun, Hui, et al.
Pubblicazione: (2025)

Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
di: Sun, Weigao, et al.
Pubblicazione: (2025)

To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models
di: Barbulescu, George-Octavian, et al.
Pubblicazione: (2024)

When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
di: You, Haoran, et al.
Pubblicazione: (2024)

Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix
di: Liang, Yingyu, et al.
Pubblicazione: (2024)

Training Tensor Attention Efficiently: From Cubic to Almost Linear Time
di: Cao, Yang, et al.
Pubblicazione: (2024)

Rethinking Reward Model Evaluation: Are We Barking up the Wrong Tree?
di: Wen, Xueru, et al.
Pubblicazione: (2024)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
di: Li, Yaxuan, et al.
Pubblicazione: (2026)

To be Continuous, or to be Discrete, Those are Bits of Questions
di: Wang, Yiran, et al.
Pubblicazione: (2024)

On Eliciting Syntax from Language Models via Hashing
di: Wang, Yiran, et al.
Pubblicazione: (2024)

Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards
di: Shen, Yiran, et al.
Pubblicazione: (2025)

Demystifying the Slash Pattern in Attention: The Role of RoPE
di: Cheng, Yuan, et al.
Pubblicazione: (2026)

MemMamba: Rethinking Memory Patterns in State Space Model
di: Wang, Youjin, et al.
Pubblicazione: (2025)