Copertina

Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	He, Zifan, Ma, Rui, Sun, Yizhou, Cong, Jason
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Hermes: Memory-Efficient Pipeline Inference for Large Models on Edge Devices
di: Han, Xueyuan, et al.
Pubblicazione: (2024)

SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting
di: Xu, Jiaming, et al.
Pubblicazione: (2025)

AdaPtis: Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models
di: Guo, Jihu, et al.
Pubblicazione: (2025)

AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure
di: The AIBrix Team, et al.
Pubblicazione: (2025)

Large Language Model Partitioning for Low-Latency Inference at the Edge
di: Kafetzis, Dimitrios, et al.
Pubblicazione: (2025)

Accelerating Large Language Model Training with Hybrid GPU-based Compression
di: Xu, Lang, et al.
Pubblicazione: (2024)

FlashMLA-ETAP: Efficient Transpose Attention Pipeline for Accelerating MLA Inference on NVIDIA H20 GPUs
di: Dege, Pengcuo, et al.
Pubblicazione: (2025)

A Survey on Large Language Model Acceleration based on KV Cache Management
di: Li, Haoyang, et al.
Pubblicazione: (2024)

DIP: Efficient Large Multimodal Model Training with Dynamic Interleaved Pipeline
di: Xue, Zhenliang, et al.
Pubblicazione: (2025)

TCM-Serve: Modality-aware Scheduling for Multimodal Large Language Model Inference
di: Papaioannou, Konstantinos, et al.
Pubblicazione: (2026)

FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
di: Liu, Xing, et al.
Pubblicazione: (2025)

Mist: Efficient Distributed Training of Large Language Models via Memory-Parallelism Co-Optimization
di: Zhu, Zhanda, et al.
Pubblicazione: (2025)

Accelerating LLM Inference with Precomputed Query Storage
di: Park, Jay H., et al.
Pubblicazione: (2025)

Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inference
di: Huang, Yafan, et al.
Pubblicazione: (2026)

Act While Thinking: Accelerating LLM Agents via Pattern-Aware Speculative Tool Execution
di: Sui, Yifan, et al.
Pubblicazione: (2026)

BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
di: Wu, Houming, et al.
Pubblicazione: (2024)

OrchMLLM: Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training
di: Zheng, Yijie, et al.
Pubblicazione: (2025)

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
di: Li, Rui, et al.
Pubblicazione: (2025)

Identifying and Mitigating Systemic Measurement Bias in Production LLM Inference Benchmarks
di: Chandrasekar, Ashok, et al.
Pubblicazione: (2026)

Leveraging Large Language Model for Intelligent Log Processing and Autonomous Debugging in Cloud AI Platforms
di: Ji, Cheng, et al.
Pubblicazione: (2025)

A-IO: Adaptive Inference Orchestration for Memory-Bound NPUs
di: Zhang, Chen, et al.
Pubblicazione: (2026)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
di: Wang, Shiju, et al.
Pubblicazione: (2025)

Optimizing Resource Allocation for Geographically-Distributed Inference by Large Language Models
di: Sun, Tingyang, et al.
Pubblicazione: (2025)

Safe and Compliant Cross-Market Trade Execution via Constrained RL and Zero-Knowledge Audits
di: Borjigin, Ailiya, et al.
Pubblicazione: (2025)

TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training
di: Wu, Houming, et al.
Pubblicazione: (2025)

Keep Your Friends Close: Leveraging Affinity Groups to Accelerate AI Inference Workflows
di: Garrett, Thiago, et al.
Pubblicazione: (2023)

Training Overhead Ratio: A Practical Reliability Metric for Large Language Model Training Systems
di: Lu, Ning, et al.
Pubblicazione: (2024)

Venus: An Efficient Edge Memory-and-Retrieval System for VLM-based Online Video Understanding
di: Ye, Shengyuan, et al.
Pubblicazione: (2025)

Scaling Performance of Large Language Model Pretraining
di: Interrante-Grant, Alexander, et al.
Pubblicazione: (2025)

Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization
di: Gao, Luyao, et al.
Pubblicazione: (2024)

Can Large Language Models Write Parallel Code?
di: Nichols, Daniel, et al.
Pubblicazione: (2024)

Hierarchical Autoscaling for Large Language Model Serving with Chiron
di: Patke, Archit, et al.
Pubblicazione: (2025)

Efficient Multi-Model Orchestration for Self-Hosted Large Language Models
di: Vangala, Bhanu Prakash, et al.
Pubblicazione: (2025)

HPC-Coder: Modeling Parallel Programs using Large Language Models
di: Nichols, Daniel, et al.
Pubblicazione: (2023)

FreeRide: Harvesting Bubbles in Pipeline Parallelism
di: Zhang, Jiashu, et al.
Pubblicazione: (2024)

Automated Planning for Optimal Data Pipeline Instantiation
di: Amado, Leonardo Rosa, et al.
Pubblicazione: (2025)

CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
di: Suo, Jiashun, et al.
Pubblicazione: (2025)

Equinox: Holistic Fair Scheduling in Serving Large Language Models
di: Wei, Zhixiang, et al.
Pubblicazione: (2025)

Accelerating a Triton Fused Kernel for W4A16 Quantized Inference with SplitK work decomposition
di: Hoque, Adnan, et al.
Pubblicazione: (2024)

Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning
di: Xu, Lang, et al.
Pubblicazione: (2025)