Copertina

HexiSeq: Accommodating Long Context Training of LLMs over Heterogeneous Hardware

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Liang, Yan, Jiang, Youhe, Yan, Ran, Yuan, Binhang, Wang, Wei, Wu, Chuan
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

HexiScale: Facilitating Large Language Model Training over Heterogeneous Hardware
di: Yan, Ran, et al.
Pubblicazione: (2024)

HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environment
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
di: Yan, Ran, et al.
Pubblicazione: (2025)

HexGen: Generative Inference of Large Language Model over Heterogeneous Environment
di: Jiang, Youhe, et al.
Pubblicazione: (2023)

Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
di: Yan, Ran, et al.
Pubblicazione: (2025)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
di: Peng, You, et al.
Pubblicazione: (2026)

Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
di: He, Guoliang, et al.
Pubblicazione: (2025)

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

Parallax: Efficient LLM Inference Service over Decentralized Environment
di: Tong, Chris, et al.
Pubblicazione: (2025)

Cascadia: An Efficient Cascade Serving System for Large Language Models
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

Heta: Distributed Training of Heterogeneous Graph Neural Networks
di: Zhong, Yuchen, et al.
Pubblicazione: (2024)

LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
di: Gu, Diandian, et al.
Pubblicazione: (2024)

DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism
di: Jiang, Chenyu, et al.
Pubblicazione: (2025)

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
di: Zhang, Shiwei, et al.
Pubblicazione: (2024)

Efficient Multi-round LLM Inference over Disaggregated Serving
di: He, Wenhao, et al.
Pubblicazione: (2026)

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training
di: Sun, Ao, et al.
Pubblicazione: (2024)

OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization
di: Hu, Rizhen, et al.
Pubblicazione: (2025)

Schedule-Level Shared-Prefix Reuse for LLM RL Training
di: Li, Pengbo, et al.
Pubblicazione: (2026)

Revisiting the Time Cost Model of AllReduce
di: Xiong, Dian, et al.
Pubblicazione: (2024)

BurstEngine: an Efficient Distributed Framework for Training Transformers on Extremely Long Sequences of over 1M Tokens
di: Sun, Ao, et al.
Pubblicazione: (2025)

Training DNN Models over Heterogeneous Clusters with Optimal Performance
di: Nie, Chengyi, et al.
Pubblicazione: (2024)

SWIFT: Expedited Failure Recovery for Large-scale DNN Training
di: Zhong, Yuchen, et al.
Pubblicazione: (2023)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
di: Strati, Foteini, et al.
Pubblicazione: (2025)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
di: Li, Suyi, et al.
Pubblicazione: (2024)

HARP: Orchestrating Automated Parallel Training on Heterogeneous GPU Clusters
di: Liang, Antian, et al.
Pubblicazione: (2025)

Efficient Long Context Fine-tuning with Chunk Flow
di: Yuan, Xiulong, et al.
Pubblicazione: (2025)

LoHan: Low-Cost High-Performance Framework to Fine-Tune 100B Model on a Consumer GPU
di: Liao, Changyue, et al.
Pubblicazione: (2024)

DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
di: Zhang, Zili, et al.
Pubblicazione: (2024)

MTGenRec: An Efficient Distributed Training System for Generative Recommendation Models in Meituan
di: Wang, Yuxiang, et al.
Pubblicazione: (2025)

DiffusionPipe: Training Large Diffusion Models with Efficient Pipelines
di: Tian, Ye, et al.
Pubblicazione: (2024)

H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
di: Tang, Ding, et al.
Pubblicazione: (2025)

HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments
di: He, Yongjun, et al.
Pubblicazione: (2025)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
di: li, Fei, et al.
Pubblicazione: (2026)

AB-Sparse: Sparse Attention with Adaptive Block Size for Accurate and Efficient Long-Context Inference
di: Liu, Di, et al.
Pubblicazione: (2026)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
di: Luo, Shuqing, et al.
Pubblicazione: (2024)

Strata: Hierarchical Context Caching for Long Context Language Model Serving
di: Xie, Zhiqiang, et al.
Pubblicazione: (2025)