Image de couverture de livre

Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bambhaniya, Abhimanyu, Raj, Ritik, Jeong, Geonhwa, Kundu, Souvik, Srinivasan, Sudarshan, Subramanian, Suvinay, Elavazhagan, Midhilesh, Kumar, Madhu, Krishna, Tushar
Format:	Preprint
Publié:	2024
Sujets:	Hardware Architecture Artificial Intelligence Distributed, Parallel, and Cluster Computing Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inference
par: Bambhaniya, Abhimanyu Rajeshkumar, et autres
Publié: (2025)

How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
par: Wu, Hanjiang, et autres
Publié: (2026)

TACOS: Topology-Aware Collective Algorithm Synthesizer for Distributed Machine Learning
par: Won, William, et autres
Publié: (2023)

HARP: A Taxonomy for Heterogeneous and Hierarchical Processors for Mixed-reuse Workloads
par: Garg, Raveesh, et autres
Publié: (2025)

CELLO: Co-designing Schedule and Hybrid Implicit/Explicit Buffer for Complex Tensor Reuse
par: Garg, Raveesh, et autres
Publié: (2023)

FengHuang: Next-Generation Memory Orchestration for AI Inferencing
par: Li, Jiamin, et autres
Publié: (2025)

Security Risks Due to Data Persistence in Cloud FPGA Platforms
par: Zhang, Zhehang, et autres
Publié: (2024)

Survey of Disaggregated Memory: Cross-layer Technique Insights for Next-Generation Datacenters
par: Wang, Jing, et autres
Publié: (2025)

Next-generation Probabilistic Computing Hardware with 3D MOSAICs, Illusion Scale-up, and Co-design
par: Srimani, Tathagata, et autres
Publié: (2024)

SwarmIO: Towards 100 Million IOPS SSD Emulation for Next-generation GPU-centric Storage Systems
par: Kim, Hyeseong, et autres
Publié: (2026)

FpgaHub: Fpga-centric Hyper-heterogeneous Computing Platform for Big Data Analytics
par: Wang, Zeke, et autres
Publié: (2025)

Characterizing CPU-Induced Slowdowns in Multi-GPU LLM Inference
par: Chung, Euijun, et autres
Publié: (2026)

Understanding Bottlenecks for Efficiently Serving LLM Inference With KV Offloading
par: Meng, William, et autres
Publié: (2025)

Automated Deep Neural Network Inference Partitioning for Distributed Embedded Systems
par: Kreß, Fabian, et autres
Publié: (2024)

Exploring the Efficiency of 3D-Stacked AI Chip Architecture for LLM Inference with Voxel
par: Liu, Yiqi, et autres
Publié: (2026)

HgPCN: A Heterogeneous Architecture for E2E Embedded Point Cloud Inference
par: Gao, Yiming, et autres
Publié: (2025)

TeraPool-SDR: An 1.89TOPS 1024 RV-Cores 4MiB Shared-L1 Cluster for Next-Generation Open-Source Software-Defined Radios
par: Zhang, Yichao, et autres
Publié: (2024)

SLIM: A Heterogeneous Accelerator for Edge Inference of Sparse Large Language Model via Adaptive Thresholding
par: Xu, Weihong, et autres
Publié: (2025)

Performance Modeling and Workload Analysis of Distributed Large Language Model Training and Inference
par: Kundu, Joyjit, et autres
Publié: (2024)

SpeedMalloc: Improving Multi-threaded Applications via a Lightweight Core for Memory Allocation
par: Li, Ruihao, et autres
Publié: (2025)

LayerDAG: A Layerwise Autoregressive Diffusion Model for Directed Acyclic Graph Generation
par: Li, Mufei, et autres
Publié: (2024)

LIBRA: Enabling Workload-aware Multi-dimensional Network Topology Optimization for Distributed Training of Large AI Models
par: Won, William, et autres
Publié: (2021)

Fusing Depthwise and Pointwise Convolutions for Efficient Inference on GPUs
par: Qararyah, Fareed, et autres
Publié: (2024)

Profiling Concurrent Vision Inference Workloads on NVIDIA Jetson -- Extended
par: Chakraborty, Abhinaba, et autres
Publié: (2025)

The Feasibility of Implementing Large-Scale Transformers on Multi-FPGA Platforms
par: Gao, Yu, et autres
Publié: (2024)

TT-Edge: A Hardware-Software Co-Design for Energy-Efficient Tensor-Train Decomposition on Edge AI
par: Kwak, Hyunseok, et autres
Publié: (2025)

The DEEP-ER project: I/O and resiliency extensions for the Cluster-Booster architecture
par: Kreuzer, Anke, et autres
Publié: (2019)

An Evaluation and Comparison of GPU Hardware and Solver Libraries for Accelerating the OPM Flow Reservoir Simulator
par: Qiu, Tong Dong, et autres
Publié: (2023)

COMET: A Framework for Modeling Compound Operation Dataflows with Explicit Collectives
par: Negi, Shubham, et autres
Publié: (2025)

MVDRAM: Enabling GeMV Execution in Unmodified DRAM for Low-Bit LLM Acceleration
par: Kubo, Tatsuya, et autres
Publié: (2025)

PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System
par: Liu, Lian, et autres
Publié: (2026)

RAPID-Graph: Recursive All-Pairs Shortest Paths Using Processing-in-Memory for Dynamic Programming on Graphs
par: Chen, Yanru, et autres
Publié: (2025)

Chopper: A Multi-Level GPU Characterization Tool & Derived Insights Into LLM Training Inefficiency
par: Kurzynski, Marco, et autres
Publié: (2025)

Towards Compute-Aware In-Switch Computing for LLMs Tensor-Parallelism on Multi-GPU Systems
par: Zhang, Chen, et autres
Publié: (2026)

Efficient deadlock avoidance for 2D mesh NoCs that use OQ or VOQ routers
par: Papaphilippou, Philippos, et autres
Publié: (2023)

DCRA: A Distributed Chiplet-based Reconfigurable Architecture for Irregular Applications
par: Orenes-Vera, Marcelo, et autres
Publié: (2023)

LFOC: A Lightweight Fairness-Oriented Cache Clustering Policy for Commodity Multicores
par: García-García, Adrián, et autres
Publié: (2024)

FlexVector: A SpMM Vector Processor with Flexible VRF for GCNs on Varying-Sparsity Graphs
par: Li, Bohan, et autres
Publié: (2026)

Sequence-Aware Split Heuristic to Mitigate SM Underutilization in FlashAttention-3 Low-Head-Count Decoding
par: Font, Martí Llopart, et autres
Publié: (2026)

iHAC: A Hybrid Cluster Architecture for Enhanced Performance and Resilience
par: Muntaka, Siddique Abubakr, et autres
Publié: (2026)