Imagen de Portada

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Holmes, Connor, Tanaka, Masahiro, Wyatt, Michael, Awan, Ammar Ahmad, Rasley, Jeff, Rajbhandari, Samyam, Aminabadi, Reza Yazdani, Qin, Heyang, Bakhtiari, Arash, Kurilenko, Lev, He, Yuxiong
Formato:	Preprint
Publicado:	2024
Materias:	Performance Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing
por: Li, Conglong, et al.
Publicado: (2022)

Scaling Vision Transformers: Evaluating DeepSpeed for Image-Centric Workloads
por: Trinh, Huy, et al.
Publicado: (2026)

Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
por: Hidayetoglu, Mert, et al.
Publicado: (2025)

Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences
por: Bekman, Stas, et al.
Publicado: (2025)

Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI
por: Rajbhandari, Samyam, et al.
Publicado: (2025)

SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation
por: Qiao, Aurick, et al.
Publicado: (2024)

MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
por: Su, Zhaoyuan, et al.
Publicado: (2026)

Fast and Accurate Causal Parallel Decoding using Jacobi Forcing
por: Hu, Lanxiang, et al.
Publicado: (2025)

A Semi Centralized Training Decentralized Execution Architecture for Multi Agent Deep Reinforcement Learning in Traffic Signal Control
por: Rezaali, Arash, et al.
Publicado: (2025)

An Empirical Investigation of Speed Patterns on S‐Curves Using Naturalistic Driving Data and Mixed Logit Model
por: Cailin Lei, et al.
Publicado: (2025)

Flow Matching for Medical Image Synthesis: Bridging the Gap Between Speed and Quality
por: Yazdani, Milad, et al.
Publicado: (2025)

OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs
por: Lee, Jaeseong, et al.
Publicado: (2025)

Speeding Up Optimization-based Motion Planning through Deep Learning
por: Tenhumberg, Johannes, et al.
Publicado: (2023)

ENTREVISTA COM RUBENS FIGUEIREDO
por: Alexey Kurilenko
Publicado: (2022)

c: Not the Speed of Light. The Speed of Reality.
por: Holub, Pavel
Publicado: (2026)

AssetGen: Deployable 3D Asset Generation at Interactive Speed
por: Wang, Dilin, et al.
Publicado: (2026)

BrainVoxGen: Deep learning framework for synthesis of Ultrasound to MRI
por: Singh, Shubham, et al.
Publicado: (2023)

Tube Loss based Deep Networks For Improving the Probabilistic Forecasting of Wind Speed
por: Anand, Pritam, et al.
Publicado: (2025)

Achieving Stable High-Speed Locomotion for Humanoid Robots with Deep Reinforcement Learning
por: Zhang, Xinming, et al.
Publicado: (2024)

Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelis
por: Lian, Xinyu, et al.
Publicado: (2024)

Tibial Strains are Sensitive to Speed, but not Grade, Perturbations During Running
por: Baggaley, Michael, et al.
Publicado: (2023)

Prompt-MII: Meta-Learning Instruction Induction for LLMs
por: Xiao, Emily, et al.
Publicado: (2025)

Deep Learning for High Speed Optical Coherence Elastography with a Fiber Scanning Endoscope
por: Neidhardt, Maximilian, et al.
Publicado: (2025)

Deep Learning Methods for Adjusting Global MFD Speed Estimations to Local Link Configurations
por: Jin, Zhixiong, et al.
Publicado: (2024)

TAG‐SPARK: Empowering High‐Speed Volumetric Imaging With Deep Learning and Spatial Redundancy
por: Yin‐Tzu Hsieh, et al.
Publicado: (2024)

Infinitely growing configurations in Emil Post's tag system problem
por: Kurilenko, Nikita V.
Publicado: (2021)

FastPersist: Accelerating Model Checkpointing in Deep Learning
por: Wang, Guanhua, et al.
Publicado: (2024)

Speed is Confidence
por: Dillon, Joshua V.
Publicado: (2026)

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design
por: Xia, Haojun, et al.
Publicado: (2024)

A Dual-Motor Actuator for Ceiling Robots with High Force and High Speed Capabilities
por: Lalonde, Ian, et al.
Publicado: (2024)

Detecting Car Speed using Object Detection and Depth Estimation: A Deep Learning Framework
por: Dasgupta, Subhasis, et al.
Publicado: (2024)

Analysis of Centrifugal Clutches in Two-Speed Automatic Transmissions with Deep Learning-Based Engagement Prediction
por: Lin, Bo-Yi, et al.
Publicado: (2024)

Short‐Term Wind Speed Prediction Model Based on Hybrid Decomposition Method and Deep Learning
por: Xueqiong Yuan, et al.
Publicado: (2025)

FastKernels: Benchmarking GPU Kernel Generation in Production
por: Oliaro, Gabriele, et al.
Publicado: (2026)

Improvement of Speed Limits: Quantum Effect on the Speed in Open Quantum Systems
por: Sekiguchi, Kotaro, et al.
Publicado: (2024)

Modeling Driver Behavior in Speed Advisory Systems: Koopman-based Approach with Online Update
por: Ozkan, Mehmet Fatih, et al.
Publicado: (2025)

Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving
por: Shao, Yuqian, et al.
Publicado: (2026)

Wind as Driver of Bird and Bat Abundance, Flight Direction, Altitude, and Speed on the North Atlantic Shelf
por: Snortland, Abigale, et al.
Publicado: (2025)

Smoothing the Speed of Light
por: Hall, Matthew
Publicado: (2025)

High-Speed Railway
Publicado: (2024)