Copertina

TokenRing: An Efficient Parallelism Framework for Infinite-Context LLMs via Bidirectional Communication

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Zongwu, Liu, Fangxin, Li, Mingshuai, Jiang, Li
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Distributed, Parallel, and Cluster Computing C.3
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

StarTrail: Concentric Ring Sequence Parallelism for Efficient Near-Infinite-Context Transformer Model Training
di: Liu, Ziming, et al.
Pubblicazione: (2024)

LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
di: Gu, Diandian, et al.
Pubblicazione: (2024)

MILLION: Mastering Long-Context LLM Inference Via Outlier-Immunized KV Product Quantization
di: Wang, Zongwu, et al.
Pubblicazione: (2025)

Argus: Token Aware Distributed LLM Inference Optimization
di: Wu, Panlong, et al.
Pubblicazione: (2025)

Communication-Efficient Serving for Video Diffusion Models with Latent Parallelism
di: Wu, Zhiyuan, et al.
Pubblicazione: (2025)

A Flexible Programmable Pipeline Parallelism Framework for Efficient DNN Training
di: Jiang, Lijuan, et al.
Pubblicazione: (2025)

Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
di: Chen, Fahao, et al.
Pubblicazione: (2024)

Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
di: He, Guoliang, et al.
Pubblicazione: (2025)

NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding
di: Chen, Jiefei, et al.
Pubblicazione: (2026)

OMP-Engineer: Bridging Syntax Analysis and In-Context Learning for Efficient Automated OpenMP Parallelization
di: Wang, Weidong, et al.
Pubblicazione: (2024)

Oases: Efficient Large-Scale Model Training on Commodity Servers via Overlapped and Automated Tensor Model Parallelism
di: Li, Shengwei, et al.
Pubblicazione: (2023)

JanusPipe: Efficient Pipeline Parallel Training for Machine Learning Interatomic Potentials
di: Wang, Hongyu, et al.
Pubblicazione: (2026)

Accelerating Microswimmer Simulations via a Heterogeneous Pipelined Parallel-in-Time Framework
di: Huang, Ruixiang, et al.
Pubblicazione: (2026)

Parallel AIG Refactoring via Conflict Breaking
di: Cai, Ye, et al.
Pubblicazione: (2024)

HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference
di: Lin, Haoran, et al.
Pubblicazione: (2025)

Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
di: Lin, Bin, et al.
Pubblicazione: (2024)

Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP
di: Zhao, Yilong, et al.
Pubblicazione: (2026)

Hecate: Unlocking Efficient Sparse Model Training via Fully Sharded Sparse Data Parallelism
di: Qing, Yuhao, et al.
Pubblicazione: (2025)

SPPO:Efficient Long-sequence LLM Training via Adaptive Sequence Pipeline Parallel Offloading
di: Chen, Qiaoling, et al.
Pubblicazione: (2025)

PUSHtap: PIM-based In-Memory HTAP with Unified Data Storage Format
di: Zhao, Yilong, et al.
Pubblicazione: (2025)

Pro-Prophet: A Systematic Load Balancing Method for Efficient Parallel Training of Large-scale MoE Models
di: Wang, Wei, et al.
Pubblicazione: (2024)

S-HPLB: Efficient LLM Attention Serving via Sparsity-Aware Head Parallelism Load Balance
di: Liu, Di, et al.
Pubblicazione: (2026)

Heimdall++: Optimizing GPU Utilization and Pipeline Parallelism for Efficient Single-Pulse Detection
di: Xia, Bingzheng, et al.
Pubblicazione: (2025)

DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism
di: Jiang, Chenyu, et al.
Pubblicazione: (2025)

Efficient Long Context Fine-tuning with Chunk Flow
di: Yuan, Xiulong, et al.
Pubblicazione: (2025)

Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization
di: Li, Haoyang, et al.
Pubblicazione: (2024)

Optimizing Long-context LLM Serving via Fine-grained Sequence Parallelism
di: Li, Cong, et al.
Pubblicazione: (2025)

HexiSeq: Accommodating Long Context Training of LLMs over Heterogeneous Hardware
di: Liang, Yan, et al.
Pubblicazione: (2026)

Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient
di: Deng, Xiaoge, et al.
Pubblicazione: (2021)

CondenseGraph: Communication-Efficient Distributed GNN Training via On-the-Fly Graph Condensation
di: Zhang, Zizhao, et al.
Pubblicazione: (2026)

Joint Training on AMD and NVIDIA GPUs
di: Hu, Jon, et al.
Pubblicazione: (2026)

Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking
di: Ghadia, Ravi, et al.
Pubblicazione: (2026)

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

BurstEngine: an Efficient Distributed Framework for Training Transformers on Extremely Long Sequences of over 1M Tokens
di: Sun, Ao, et al.
Pubblicazione: (2025)

Efficient Counting and Simulation in Content-Oblivious Rings
di: Chalopin, Jérémie, et al.
Pubblicazione: (2026)

ZeroPP: Unleashing Exceptional Parallelism Efficiency through Tensor-Parallelism-Free Methodology
di: Tang, Ding, et al.
Pubblicazione: (2024)

GRNND: A GPU-Parallel Relative NN-Descent Algorithm for Efficient Approximate Nearest Neighbor Graph Construction
di: Li, Xiang, et al.
Pubblicazione: (2025)

Scalable Genomic Context Analysis with GCsnap2 on HPC Clusters
di: Krummenacher, Reto, et al.
Pubblicazione: (2025)

Memory Efficient and Staleness Free Pipeline Parallel DNN Training Framework with Improved Convergence Speed
di: Dutta, Ankita, et al.
Pubblicazione: (2025)

Enhancing ASIC Technology Mapping via Parallel Supergate Computing
di: Cai, Ye, et al.
Pubblicazione: (2024)