Copertina

HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhao, Xuanlei, Jia, Bin, Zhou, Haotian, Liu, Ziming, Cheng, Shenggan, You, Yang
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Performance Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference
di: Zhao, Xuanlei, et al.
Pubblicazione: (2024)

HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
di: Zhang, Geng, et al.
Pubblicazione: (2025)

StarTrail: Concentric Ring Sequence Parallelism for Efficient Near-Infinite-Context Transformer Model Training
di: Liu, Ziming, et al.
Pubblicazione: (2024)

DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers
di: Zhao, Xuanlei, et al.
Pubblicazione: (2024)

HybridGen: Efficient LLM Generative Inference via CPU-GPU Hybrid Computing
di: Lin, Mao, et al.
Pubblicazione: (2026)

Fine-Grained Energy Prediction For Parallellized LLM Inference With PIE-P
di: Dutt, Anurag, et al.
Pubblicazione: (2025)

Collaborative Processing for Multi-Tenant Inference on Memory-Constrained Edge TPUs
di: Ng, Nathan, et al.
Pubblicazione: (2026)

CoFormer: Collaborating with Heterogeneous Edge Devices for Scalable Transformer Inference
di: Xu, Guanyu, et al.
Pubblicazione: (2025)

Opt4GPTQ: Co-Optimizing Memory and Computation for 4-bit GPTQ Quantized LLM Inference on Heterogeneous Platforms
di: Zhang, Yaozheng, et al.
Pubblicazione: (2025)

Parallel I/O Characterization and Optimization on Large-Scale HPC Systems: A 360-Degree Survey
di: Ather, Hammad, et al.
Pubblicazione: (2024)

On Orchestrating Parallel Broadcasts for Distributed Ledgers
di: Sheng, Peiyao, et al.
Pubblicazione: (2024)

Fault-Tolerant Hybrid-Parallel Training at Scale with Reliable and Efficient In-memory Checkpointing
di: Wang, Yuxin, et al.
Pubblicazione: (2023)

Automated Programmatic Performance Analysis of Parallel Programs
di: Cankur, Onur, et al.
Pubblicazione: (2024)

Optimal Parallel Scheduling under Concave Speedup Functions
di: Li, Chengzhang, et al.
Pubblicazione: (2025)

Recorder: Comprehensive Parallel I/O Tracing and Analysis
di: Wang, Chen, et al.
Pubblicazione: (2025)

Resource Management Schemes for Cloud-Native Platforms with Computing Containers of Docker and Kubernetes
di: Mao, Ying, et al.
Pubblicazione: (2020)

Can Large Language Models Predict Parallel Code Performance?
di: Bolet, Gregory, et al.
Pubblicazione: (2025)

THAPI: Tracing Heterogeneous APIs
di: Bekele, Solomon, et al.
Pubblicazione: (2025)

ParaLog: Consistent Host-side Logging for Parallel Checkpoints
di: Chien, Steven W. D., et al.
Pubblicazione: (2024)

Cache Blocking of Distributed-Memory Parallel Matrix Power Kernels
di: Lacey, Dane C., et al.
Pubblicazione: (2024)

Automated Calibration of Parallel and Distributed Computing Simulators: A Case Study
di: McDonald, Jesse, et al.
Pubblicazione: (2024)

Performance Impact of Containerized METADOCK 2 on Heterogeneous Platforms
di: Banegas-Luna, Antonio Jesús, et al.
Pubblicazione: (2025)

Understanding Power Consumption Metric on Heterogeneous Memory Systems
di: Proaño, Andrès Rubio, et al.
Pubblicazione: (2024)

An Empirical Characterization of Outages and Incidents in Public Services for Large Language Models
di: Chu, Xiaoyu, et al.
Pubblicazione: (2025)

Matryoshka: Optimization of Dynamic Diverse Quantum Chemistry Systems via Elastic Parallelism Transformation
di: Wang, Tuowei, et al.
Pubblicazione: (2024)

CARAT: Client-Side Adaptive RPC and Cache Co-Tuning for Parallel File Systems
di: Rashid, Md Hasanur, et al.
Pubblicazione: (2026)

Characterizing Adaptive Mesh Refinement on Heterogeneous Platforms with Parthenon-VIBE
di: Poptani, Akash, et al.
Pubblicazione: (2025)

AcceleratedKernels.jl: Cross-Architecture Parallel Algorithms from a Unified, Transpiled Codebase
di: Nicusan, Andrei-Leonard, et al.
Pubblicazione: (2025)

Cloud Resource Allocation with Convex Optimization
di: Boghani, Shayan, et al.
Pubblicazione: (2025)

Enhancing Performance Insight at Scale: A Heterogeneous Framework for Exascale Diagnostics
di: Grbic, Dragana
Pubblicazione: (2026)

FalconFS: Distributed File System for Large-Scale Deep Learning Pipeline
di: Xu, Jingwei, et al.
Pubblicazione: (2025)

ADELIA: Automatic Differentiation for Efficient Laplace Inference Approximations
di: Boudaoud, Afif, et al.
Pubblicazione: (2026)

Optimal Configuration of API Resources in Cloud Native Computing
di: Truyen, Eddy, et al.
Pubblicazione: (2025)

A Practical Two-Stage Framework for GPU Resource and Power Prediction in Heterogeneous HPC Systems
di: Oztop, Beste, et al.
Pubblicazione: (2026)

DIAL: Decentralized I/O AutoTuning via Learned Client-side Local Metrics for Parallel File System
di: Rashid, Md Hasanur, et al.
Pubblicazione: (2026)

Comparing the Performance of Heterogeneous Conjugate Gradient and Cholesky Solvers on Various Hardware Using SYCL
di: Thüring, Tim, et al.
Pubblicazione: (2026)

LMDeploy Accelerates Mixed-Precision LLM Inference with TurboMind
di: Zhang, Li, et al.
Pubblicazione: (2025)

Understanding Inference Scaling for LLMs: Bottlenecks, Trade-offs, and Performance Principles
di: Arif, Moiz, et al.
Pubblicazione: (2026)

Data-Driven Analysis to Understand GPU Hardware Resource Usage of Optimizations
di: Islam, Tanzima Z., et al.
Pubblicazione: (2024)

Ariel-ML: Computing Parallelization with Embedded Rust for Neural Networks on Heterogeneous Multi-core Microcontrollers
di: Huang, Zhaolan, et al.
Pubblicazione: (2025)