Cover Image

Improving Automatic Parallel Training via Balanced Memory Workload Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yujie, Jiang, Youhe, Miao, Xupeng, Fu, Fangcheng, Zhu, Shenhan, Nie, Xiaonan, Tu, Yaofeng, Cui, Bin
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Databases Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
by: Nie, Xiaonan, et al.
Published: (2024)

HexiScale: Facilitating Large Language Model Training over Heterogeneous Hardware
by: Yan, Ran, et al.
Published: (2024)

Spindle: Efficient Distributed Training of Multi-Task Large Models via Wavefront Scheduling
by: Wang, Yujie, et al.
Published: (2024)

Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization
by: Li, Haoyang, et al.
Published: (2024)

OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration
by: Jiang, Youhe, et al.
Published: (2026)

Galvatron: An Automatic Distributed System for Efficient Foundation Model Training
by: Liu, Xinyi, et al.
Published: (2025)

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization
by: Jiang, Youhe, et al.
Published: (2026)

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
by: Jiang, Youhe, et al.
Published: (2025)

FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism
by: Wang, Yujie, et al.
Published: (2024)

TierBase: A Workload-Driven Cost-Optimized Key-Value Store
by: Shen, Zhitao, et al.
Published: (2025)

LobRA: Multi-tenant Fine-tuning over Heterogeneous Data
by: Lin, Sheng, et al.
Published: (2025)

Efficient Multi-round LLM Inference over Disaggregated Serving
by: He, Wenhao, et al.
Published: (2026)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
by: Jiang, Youhe, et al.
Published: (2025)

SEE++: Evolving Snowpark Execution Environment for Modern Workloads
by: Jain, Gaurav, et al.
Published: (2025)

Parallel R-tree-based Spatial Query Processing on a Commercial Processing-in-Memory System
by: Jannat, Tasmia, et al.
Published: (2026)

Hetu v2: A General and Scalable Deep Learning System with Hierarchical and Heterogeneous Single Program Multiple Data Annotations
by: Li, Haoyang, et al.
Published: (2025)

Balancing Fairness and Performance in Multi-User Spark Workloads with Dynamic Scheduling (extended version)
by: Kažemaks, Dāvis, et al.
Published: (2025)

OASIS: Object-based Analytics Storage for Intelligent SQL Query Offloading in Scientific Tabular Workloads
by: Hwang, Soon, et al.
Published: (2025)

Zeppelin: Balancing Variable-length Workloads in Data Parallel Large Model Training
by: Chen, Chang, et al.
Published: (2025)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
by: Wang, Shiju, et al.
Published: (2025)

TridentServe: A Stage-level Serving System for Diffusion Pipelines
by: Xia, Yifei, et al.
Published: (2025)

PDSP-Bench: A Benchmarking System for Parallel and Distributed Stream Processing
by: Agnihotri, Pratyush, et al.
Published: (2025)

Learning from the Past: Adaptive Parallelism Tuning for Stream Processing Systems
by: Han, Yuxing, et al.
Published: (2025)

Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment
by: Li, Haoyang, et al.
Published: (2024)

DEX: Scalable Range Indexing on Disaggregated Memory [Extended Version]
by: Lu, Baotong, et al.
Published: (2024)

CIDER: Boosting Memory-Disaggregated Key-Value Stores with Pessimistic Synchronization
by: Du, Yuxuan, et al.
Published: (2026)

Next Generation Cloud-native In-Memory Stores: From Redis to Valkey and Beyond
by: Rosensch"old, Carl-Johan Fauvelle Munck af, et al.
Published: (2025)

Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics
by: Jiang, Youhe, et al.
Published: (2026)

Vortex: Overcoming Memory Capacity Limitations in GPU-Accelerated Large-Scale Data Analytics
by: Yuan, Yichao, et al.
Published: (2025)

Enabling Secure and Ephemeral AI Workloads in Data Mesh Environments
by: Patel, Chinkit, et al.
Published: (2025)

Embedding Samples Dispatching for Recommendation Model Training in Edge Environments
by: Li, Guopeng, et al.
Published: (2025)

Cascadia: An Efficient Cascade Serving System for Large Language Models
by: Jiang, Youhe, et al.
Published: (2025)

Batch Query Processing and Optimization for Agentic Workflows
by: Shen, Junyi, et al.
Published: (2025)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
by: Liu, Xinyi, et al.
Published: (2026)

Optimizing Distributed Protocols with Query Rewrites [Technical Report]
by: Chu, David, et al.
Published: (2024)

Push Down Optimization for Distributed Multi Cloud Data Integration
by: Kodali, Ravi Kiran, et al.
Published: (2026)

Efficient Candidate-Free R-S Set Similarity Joins with Filter-and-Verification Trees on MapReduce
by: Feng, Yuhong, et al.
Published: (2025)

Accelerating Heterogeneous Tensor Parallelism via Flexible Workload Control
by: Wang, Zhigang, et al.
Published: (2024)

SYMPHONY: Improving Memory Management for LLM Inference Workloads
by: Agarwal, Saurabh, et al.
Published: (2024)

Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing
by: Lyu, Chenghao, et al.
Published: (2022)