Image de couverture de livre

ZeCO: Zero Communication Overhead Sequence Parallelism for Linear Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chou, Yuhong, Liu, Zehao, Zhu, Ruijie, Wan, Xinyi, Li, Tianjian, Chu, Congying, Liu, Qian, Wu, Jibin, Ma, Zejun
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map
par: Chou, Yuhong, et autres
Publié: (2024)

AMSP: Reducing Communication Overhead of ZeRO for Efficient LLM Training
par: Chen, Qiaoling, et autres
Publié: (2023)

Scaling Linear Attention with Sparse State Expansion
par: Pan, Yuqi, et autres
Publié: (2025)

Linear Attention Sequence Parallelism
par: Sun, Weigao, et autres
Publié: (2024)

Zero Bubble Pipeline Parallelism
par: Qi, Penghui, et autres
Publié: (2023)

MDN: Parallelizing Stepwise Momentum for Delta Linear Attention
par: Huang, Yulong, et autres
Publié: (2026)

ZePo: Zero-Shot Portrait Stylization with Faster Sampling
par: Liu, Jin, et autres
Publié: (2024)

IML-Spikeformer: Input-aware Multi-Level Spiking Transformer for Speech Processing
par: Song, Zeyang, et autres
Publié: (2025)

ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attention
par: Liao, Bencheng, et autres
Publié: (2024)

Stochastic Attention: Connectome-Inspired Randomized Routing for Expressive Linear-Time Attention
par: Jin, Zehao, et autres
Publié: (2026)

Gated Slot Attention for Efficient Linear-Time Sequence Modeling
par: Zhang, Yu, et autres
Publié: (2024)

LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid
par: Sun, Weigao, et autres
Publié: (2025)

S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models
par: Young, Jack
Publié: (2026)

Low Overhead Beam Alignment for Mobile Millimeter Channel Based on Continuous-Time Prediction
par: Lin, Huang-Chou, et autres
Publié: (2023)

ZeQR: Zero-shot Query Reformulation for Conversational Search
par: Yang, Dayu, et autres
Publié: (2023)

PMSN: A Parallel Multi-compartment Spiking Neuron for Multi-scale Temporal Processing
par: Chen, Xinyi, et autres
Publié: (2024)

A Systematic Analysis of Hybrid Linear Attention
par: Wang, Dustin, et autres
Publié: (2025)

SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
par: Zeng, Weihao, et autres
Publié: (2025)

ZeBROD: Zero-Retraining Based Recognition and Object Detection Framework
par: Hidayatullah, Priyanto, et autres
Publié: (2025)

HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
par: Zhang, Geng, et autres
Publié: (2025)

PEAR: Position-Embedding-Agnostic Attention Re-weighting Enhances Retrieval-Augmented Generation with Zero Inference Overhead
par: Tan, Tao, et autres
Publié: (2024)

HEAR: An EEG Foundation Model with Heterogeneous Electrode Adaptive Representation
par: Chen, Zhige, et autres
Publié: (2025)

AsyncHZP: Hierarchical ZeRO Parallelism with Asynchronous Scheduling for Scalable LLM Training
par: Bai, Huawei, et autres
Publié: (2025)

ZeST: an LLM-based Zero-Shot Traversability Navigation for Unknown Environments
par: Gummadi, Shreya, et autres
Publié: (2025)

ZeST: Zero-Shot Material Transfer from a Single Image
par: Cheng, Ta-Ying, et autres
Publié: (2024)

Bridging the Semantic Gap: An Ensemble Learning Framework With Textual Topic‐Raw Financial Feature Fusion to Enhance Fraud Detection in Chinese Markets
par: Congying Wei, et autres
Publié: (2025)

Zé Dirceu Memórias
par: Fernando Tadeu Germinatti
Publié: (2020)

ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents
par: Liu, Tianjian, et autres
Publié: (2025)

Balancing Pipeline Parallelism with Vocabulary Parallelism
par: Yeung, Man Tsung, et autres
Publié: (2024)

Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
par: Wang, Xinyu, et autres
Publié: (2025)

IMELL Cut Elimination with Linear Overhead
par: Accattoli, Beniamino, et autres
Publié: (2024)

GLU Attention Improve Transformer
par: Wang, Zehao
Publié: (2025)

ZeFaV: Boosting Large Language Models for Zero-shot Fact Verification
par: Luu, Son T., et autres
Publié: (2024)

RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping
par: Chen, Yuzong, et autres
Publié: (2025)

Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM
par: Fan, Zehao, et autres
Publié: (2025)

ZeroS: Zero-Sum Linear Attention for Efficient Transformers
par: Lu, Jiecheng, et autres
Publié: (2026)

FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism
par: Wang, Yujie, et autres
Publié: (2024)

GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning
par: Deng, Zehao, et autres
Publié: (2026)

ZePT: Zero-Shot Pan-Tumor Segmentation via Query-Disentangling and Self-Prompting
par: Jiang, Yankai, et autres
Publié: (2023)

Pipeline Parallelism with Controllable Memory
par: Qi, Penghui, et autres
Publié: (2024)