Imagen de Portada

Accelerating a Triton Fused Kernel for W4A16 Quantized Inference with SplitK work decomposition

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Hoque, Adnan, Wright, Less, Yang, Chih-Chieh, Srivatsa, Mudhakar, Ganti, Raghu
Formato:	Preprint
Publicado:	2024
Materias:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

HadaCore: Tensor Core Accelerated Hadamard Transform Kernel
por: Agarwal, Krish, et al.
Publicado: (2024)

TP-Aware Dequantization
por: Hoque, Adnan, et al.
Publicado: (2024)

Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
por: Zheng, Size, et al.
Publicado: (2025)

tritonBLAS: Triton-based Analytical Approach for GEMM Kernel Parameter Selection
por: Swann, Ryan, et al.
Publicado: (2025)

Liger Kernel: Efficient Triton Kernels for LLM Training
por: Hsu, Pin-Lun, et al.
Publicado: (2024)

Popcorn: Accelerating Kernel K-means on GPUs through Sparse Linear Algebra
por: Bellavita, Julian, et al.
Publicado: (2025)

The Fused Kernel Library: A C++ API to Develop Highly-Efficient GPU Libraries
por: Amoros, Oscar, et al.
Publicado: (2025)

Enabling Dynamic Sparsity in Quantized LLM Inference
por: Wang, Rongxiang, et al.
Publicado: (2025)

NineToothed: A Triton-Based High-Level Domain-Specific Language for Machine Learning
por: Huang, Jiacheng, et al.
Publicado: (2025)

W4A16 Mixed-Precision Matrix Multiplication on Decoupled Architecture: Kernel Design and Memory Bottleneck Analysis for Ascend NPUs
por: He, Yuanhong, et al.
Publicado: (2026)

Quasar: Quantized Self-Speculative Acceleration for Rapid Inference via Memory-Efficient Verification
por: Huang, Guang, et al.
Publicado: (2026)

Iris: First-Class Multi-GPU Programming Experience in Triton
por: Awad, Muhammad, et al.
Publicado: (2025)

Syncopate: Efficient Multi-GPU AI Kernels via Automatic Chunk-Centric Compute-Communication Overlap
por: Qiang, Xinwei, et al.
Publicado: (2026)

Where to Split? A Pareto-Front Analysis of DNN Partitioning for Edge Inference
por: Masud, Adiba, et al.
Publicado: (2026)

Understanding the Performance and Power of LLM Inferencing on Edge Accelerators
por: Arya, Mayank, et al.
Publicado: (2025)

Accelerating Distributed MoE Training and Inference with Lina
por: Li, Jiamin, et al.
Publicado: (2022)

Collaborative Inference Acceleration with Non-Penetrative Tensor Partitioning
por: Liu, Zhibang, et al.
Publicado: (2025)

Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
por: Srivatsa, Vikranth, et al.
Publicado: (2026)

Split CNN Inference on Networked Microcontrollers
por: Lu, Junyu, et al.
Publicado: (2026)

VQ-LLM: High-performance Code Generation for Vector Quantization Augmented LLM Inference
por: Liu, Zihan, et al.
Publicado: (2025)

Performance Characterization of Containerized DNN Training and Inference on Edge Accelerators
por: K., Prashanthi S., et al.
Publicado: (2023)

Accelerating OpenPangu Inference on NPU via Speculative Decoding
por: Dai, Yuntao, et al.
Publicado: (2026)

Fulcrum: Optimizing Concurrent DNN Training and Inferencing on Edge Accelerators
por: K., Prashanthi S., et al.
Publicado: (2025)

S2M3: Split-and-Share Multi-Modal Models for Distributed Multi-Task Inference on the Edge
por: Yoon, JinYi, et al.
Publicado: (2025)

Minions: Accelerating Large Language Model Inference with Aggregated Speculative Execution
por: Wang, Siqi, et al.
Publicado: (2024)

Evaluating Multi-Instance DNN Inferencing on Multiple Accelerators of an Edge Device
por: Tayal, Mumuksh, et al.
Publicado: (2025)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
por: Wang, Zhibin, et al.
Publicado: (2025)

SPIN: Accelerating Large Language Model Inference with Heterogeneous Speculative Models
por: Chen, Fahao, et al.
Publicado: (2025)

RTop-K: Ultra-Fast Row-Wise Top-K Selection for Neural Network Acceleration on GPUs
por: Xie, Xi, et al.
Publicado: (2024)

Fusing Depthwise and Pointwise Convolutions for Efficient Inference on GPUs
por: Qararyah, Fareed, et al.
Publicado: (2024)

LIME:Accelerating Collaborative Lossless LLM Inference on Memory-Constrained Edge Devices
por: Sun, Mingyu, et al.
Publicado: (2025)

InferCept: Efficient Intercept Support for Augmented Large Language Model Inference
por: Abhyankar, Reyna, et al.
Publicado: (2024)

Salted Inference: Enhancing Privacy while Maintaining Efficiency of Split Inference in Mobile Computing
por: Malekzadeh, Mohammad, et al.
Publicado: (2023)

Accelerate Intermittent Deep Inference
por: Zhang, Ziliang
Publicado: (2024)

AcceLLM: Accelerating LLM Inference using Redundancy for Load Balancing and Data Locality
por: Bournias, Ilias, et al.
Publicado: (2024)

Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization
por: Gao, Luyao, et al.
Publicado: (2024)

Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement
por: Wu, Tian, et al.
Publicado: (2025)

AcceleratedKernels.jl: Cross-Architecture Parallel Algorithms from a Unified, Transpiled Codebase
por: Nicusan, Andrei-Leonard, et al.
Publicado: (2025)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
por: Chen, Liangkun, et al.
Publicado: (2025)

FlexPie: Accelerate Distributed Inference on Edge Devices with Flexible Combinatorial Optimization[Technical Report]
por: Zhang, Runhua, et al.
Publicado: (2025)