Image de couverture de livre

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liu, Man, Liu, Xingchen, Tian, Xingjian, Lu, Bing, Lyu, Shengkay, Yin, Shengquan, Huang, Wenjing, Wei, Zheng, Zhao, Hairui, Tan, Guangming, Tao, Dingwen
Format:	Preprint
Publié:	2026
Sujets:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
par: Liu, Zedong, et autres
Publié: (2026)

ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelism
par: Liu, Zedong, et autres
Publié: (2025)

Amoeba: Runtime Tensor Parallel Transformation for LLM Inference Services
par: Chen, Haoyu, et autres
Publié: (2025)

Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
par: Srivatsa, Vikranth, et autres
Publié: (2026)

Synergistic Tensor and Pipeline Parallelism
par: Qi, Mengshi, et autres
Publié: (2025)

Overcoming Memory Constraints in Quantum Circuit Simulation with a High-Fidelity Compression Framework
par: Zhang, Boyuan, et autres
Publié: (2024)

TSUE: A Two-Stage Data Update Method for an Erasure Coded Cluster File System
par: Wei, Zheng, et autres
Publié: (2025)

AnchorTP: Resilient LLM Inference with State-Preserving Elastic Tensor Parallelism
par: Xu, Wendong, et autres
Publié: (2025)

TensorHub: Scalable and Elastic Weight Transfer for LLM RL Training
par: Ye, Chenhao, et autres
Publié: (2026)

NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism
par: Ai, Xin, et autres
Publié: (2024)

Oases: Efficient Large-Scale Model Training on Commodity Servers via Overlapped and Automated Tensor Model Parallelism
par: Li, Shengwei, et autres
Publié: (2023)

Federated Learning Using Coupled Tensor Train Decomposition
par: Zhang, Xiangtao, et autres
Publié: (2024)

JanusPipe: Efficient Pipeline Parallel Training for Machine Learning Interatomic Potentials
par: Wang, Hongyu, et autres
Publié: (2026)

Nonuniform-Tensor-Parallelism: Mitigating GPU failure impact for Scaled-up LLM Training
par: Arfeen, Daiyaan, et autres
Publié: (2025)

Accelerating Heterogeneous Tensor Parallelism via Flexible Workload Control
par: Wang, Zhigang, et autres
Publié: (2024)

ZeroPP: Unleashing Exceptional Parallelism Efficiency through Tensor-Parallelism-Free Methodology
par: Tang, Ding, et autres
Publié: (2024)

Minimizing Communication for Parallel Symmetric Tensor Times Same Vector Computation
par: Daas, Hussam Al, et autres
Publié: (2025)

CCL-D: A High-Precision Diagnostic System for Slow and Hang Anomalies in Large-Scale Model Training
par: Gu, Yida, et autres
Publié: (2026)

Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference
par: Shyam, Vasu, et autres
Publié: (2026)

vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving
par: Xu, Jiale, et autres
Publié: (2024)

TACO: A Toolsuite for the Verification of Threshold Automata
par: Eichler, Paul, et autres
Publié: (2026)

Collaborative Inference Acceleration with Non-Penetrative Tensor Partitioning
par: Liu, Zhibang, et autres
Publié: (2025)

Pipelined Dense Symmetric Eigenvalue Decomposition on Multi-GPU Architectures
par: Wang, Hansheng, et autres
Publié: (2025)

ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs
par: Yang, Jinwu, et autres
Publié: (2026)

MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training
par: Zhao, Pinxue, et autres
Publié: (2024)

Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
par: Zhao, Long, et autres
Publié: (2026)

A Flexible Programmable Pipeline Parallelism Framework for Efficient DNN Training
par: Jiang, Lijuan, et autres
Publié: (2025)

Bandwidth-Aware and Cost-Efficient Pipeline Parallel Scheduling in Geo-Distributed LLM Training
par: Zhang, Han, et autres
Publié: (2026)

ShardTensor: Domain Parallelism for Scientific Machine Learning
par: Adams, Corey, et autres
Publié: (2026)

ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training
par: Kang, Xueze, et autres
Publié: (2025)

SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training
par: Jia, Jinda, et autres
Publié: (2024)

10Cache: Heterogeneous Resource-Aware Tensor Caching and Migration for LLM Training
par: Afroz, Sabiha, et autres
Publié: (2025)

cuFastTuckerPlus: A Stochastic Parallel Sparse FastTucker Decomposition Using GPU Tensor Cores
par: Li, Zixuan, et autres
Publié: (2024)

Scaling State-Space Models on Multiple GPUs with Tensor Parallelism
par: Dutt, Anurag, et autres
Publié: (2026)

Parallelizing Large-Scale Tensor Network Contraction on Multiple GPUs
par: Pan, Feng, et autres
Publié: (2026)

CAT: Cellular Automata on Tensor cores
par: Navarro, Cristóbal A., et autres
Publié: (2024)

Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP
par: Zhao, Yilong, et autres
Publié: (2026)

Gensor: A Graph-based Construction Tensor Compilation Method for Deep Learning
par: Liu, Hangda, et autres
Publié: (2025)

On some orthogonalization schemes in Tensor Train format
par: Coulaud, Olivier, et autres
Publié: (2022)

Fast and Scalable Mixed Precision Euclidean Distance Calculations Using GPU Tensor Cores
par: Curless, Brian, et autres
Publié: (2025)