Image de couverture de livre

Accelerating Large Language Model Training with Hybrid GPU-based Compression

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Xu, Lang, Anthony, Quentin, Zhou, Qinghua, Alnaasan, Nawras, Gulhane, Radha R., Shafi, Aamir, Subramoni, Hari, Panda, Dhabaleswar K.
Format:	Preprint
Publié:	2024
Sujets:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning
par: Xu, Lang, et autres
Publié: (2025)

Characterizing Communication Patterns in Distributed Large Language Model Inference
par: Xu, Lang, et autres
Publié: (2025)

Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference
par: Yao, Jinghan, et autres
Publié: (2024)

Demystifying the Communication Characteristics for Distributed Transformer Models
par: Anthony, Quentin, et autres
Publié: (2024)

The Case for Co-Designing Model Architectures with Hardware
par: Anthony, Quentin, et autres
Publié: (2024)

From Skew to Symmetry: Node-Interconnect Multi-Path Balancing with Execution-time Planning for Modern GPU Clusters
par: Yao, Jinghan, et autres
Publié: (2026)

Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer
par: Yao, Jinghan, et autres
Publié: (2024)

gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters
par: Huang, Jiajun, et autres
Publié: (2023)

GPU-Accelerated Distributed QAOA on Large-scale HPC Ecosystems
par: Xu, Zhihao, et autres
Publié: (2025)

Comparative Study of Large Language Model Architectures on Frontier
par: Yin, Junqi, et autres
Publié: (2024)

Accelerating Biclique Counting on GPU
par: Qiu, Linshan, et autres
Publié: (2024)

Speeding up Local Optimization in Vehicle Routing with Tensor-based GPU Acceleration
par: Lei, Zhenyu, et autres
Publié: (2025)

HC-SpMM: Accelerating Sparse Matrix-Matrix Multiplication for Graphs with Hybrid GPU Cores
par: Li, Zhonggen, et autres
Publié: (2024)

GPU Accelerated Sparse Cholesky Factorization
par: Karsavuran, M. Ozan, et autres
Publié: (2024)

CRIUgpu: Transparent Checkpointing of GPU-Accelerated Workloads
par: Stoyanov, Radostin, et autres
Publié: (2025)

GPU-Accelerated Batch-Dynamic Subgraph Matching
par: Qiu, Linshan, et autres
Publié: (2024)

Large Scale Multi-GPU Based Parallel Traffic Simulation for Accelerated Traffic Assignment and Propagation
par: Jiang, Xuan, et autres
Publié: (2024)

Accelerating Intra-Node GPU-to-GPU Communication Through Multi-Path Transfers with CUDA Graphs
par: Sojoodi, Amirhossein, et autres
Publié: (2026)

An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters
par: Zhang, Mingjun, et autres
Publié: (2025)

Accelerating Sparse MTTKRP for Small Tensor Decomposition on GPU
par: Wijeratne, Sasindu, et autres
Publié: (2025)

PICO: Accelerating All k-Core Paradigms on GPU
par: Zhao, Chen, et autres
Publié: (2024)

GPZ: GPU-Accelerated Lossy Compressor for Particle Data
par: Li, Ruoyu, et autres
Publié: (2025)

Efficient Accelerated Graph Edit Distance Computation on GPU
par: Dabah, Adel, et autres
Publié: (2026)

A Preliminary Study on Accelerating Simulation Optimization with GPU Implementation
par: He, Jinghai, et autres
Publié: (2024)

PilotANN: Memory-Bounded GPU Acceleration for Vector Search
par: Gui, Yuntao, et autres
Publié: (2025)

A Practical GPU-Accelerated Implementation of Orthogonal Matching Pursuit
par: Lubonja, Ariel, et autres
Publié: (2024)

GPU Acceleration for Faster Evolutionary Spatial Cyclic Game Systems
par: Sinadjan, Louie
Publié: (2025)

Accelerating Drug Discovery in AutoDock-GPU with Tensor Cores
par: Schieffer, Gabin, et autres
Publié: (2024)

SOLANET: Distributed Neighbor Graph Construction on GPU-Accelerated Systems
par: Iwabuchi, Keita, et autres
Publié: (2026)

NCCLZ: Compression-Enabled GPU Collectives with Decoupled Quantization and Entropy Coding
par: Wang, Jiamin, et autres
Publié: (2026)

Towards Affordable, Adaptive and Automatic GNN Training on CPU-GPU Heterogeneous Platforms
par: Qiao, Tong, et autres
Publié: (2025)

SPIN: Accelerating Large Language Model Inference with Heterogeneous Speculative Models
par: Chen, Fahao, et autres
Publié: (2025)

GMLake: Efficient and Transparent GPU Memory Defragmentation for Large-scale DNN Training with Virtual Memory Stitching
par: Guo, Cong, et autres
Publié: (2024)

A Unified CPU-GPU Protocol for GNN Training
par: Lin, Yi-Chien, et autres
Publié: (2024)

Dataflow-Oriented Classification and Performance Analysis of GPU-Accelerated Homomorphic Encryption
par: Nozaki, Ai, et autres
Publié: (2026)

A GPU Accelerated Temporal Window-Based Random Walk Sampler
par: Salehin, Md Ashfaq, et autres
Publié: (2026)

GPU-Accelerated Modified Bessel Function of the Second Kind for Gaussian Processes
par: Geng, Zipei, et autres
Publié: (2025)

GPU-Accelerated Selected Basis Diagonalization with Thrust for SQD-based Algorithms
par: Doi, Jun, et autres
Publié: (2026)

HAS-GPU: Efficient Hybrid Auto-scaling with Fine-grained GPU Allocation for SLO-aware Serverless Inferences
par: Gu, Jianfeng, et autres
Publié: (2025)

Towards Scalable GPU-Accelerated SNN Training via Temporal Fusion
par: Li, Yanchen, et autres
Publié: (2024)