Cover Image

Accelerating Sparse Transformer Inference on GPU

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dai, Wenhao, Deng, Haodong, Rong, Mengfei, Yang, Xinyu, Liu, Hongyu, Liu, Fangxin, Yang, Hailong, Cao, Qianwen, Sun, Qingxiao
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operations
by: Zhao, Zhixiong, et al.
Published: (2025)

ESACT: An End-to-End Sparse Accelerator for Compute-Intensive Transformers via Local Similarity
by: Liu, Hongxiang, et al.
Published: (2025)

DASH: Input-Aware Dynamic Layer Skipping for Efficient LLM Inference with Markov Decision Policies
by: Yang, Ning, et al.
Published: (2025)

GPU-Accelerated Optimization of Transformer-Based Neural Networks for Real-Time Inference
by: Mukherjee, Soutrik, et al.
Published: (2026)

xGR: Efficient Generative Recommendation Serving at Scale
by: Sun, Qingxiao, et al.
Published: (2025)

Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference
by: Sui, Yueyuan, et al.
Published: (2026)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
by: Zhang, Haihan, et al.
Published: (2024)

HGCA: Hybrid GPU-CPU Attention for Long Context LLM Inference
by: Deng, Weishu, et al.
Published: (2025)

A GPU-Accelerated Bi-linear ADMM Algorithm for Distributed Sparse Machine Learning
by: Olama, Alireza, et al.
Published: (2024)

Nested Slice Sampling: Vectorized Nested Sampling for GPU-Accelerated Inference
by: Yallup, David, et al.
Published: (2026)

SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformers
by: Liu, Joseph, et al.
Published: (2024)

FlashOmni: A Unified Sparse Attention Engine for Diffusion Transformers
by: Qiao, Liang, et al.
Published: (2025)

Linear Trading Position with Sparse Spectrum
by: Lai, Zhao-Rong, et al.
Published: (2025)

ASTRA: Communication-Efficient Acceleration for Multi-Device Transformer Inference
by: Liu, Xiao, et al.
Published: (2025)

Estimation and Inference in Distributional Reinforcement Learning
by: Zhang, Liangyu, et al.
Published: (2023)

On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery
by: Liu, Renpu, et al.
Published: (2024)

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
by: Liu, Enshu, et al.
Published: (2024)

An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference
by: Yao, Feiyu, et al.
Published: (2026)

SparseDVFS: Sparse-Aware DVFS for Energy-Efficient Edge Inference
by: Zhang, Ziyang, et al.
Published: (2026)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
by: Zhu, Qianchao, et al.
Published: (2024)

GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference
by: Zeng, Chao, et al.
Published: (2024)

Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
by: Fan, Zehao, et al.
Published: (2025)

COBRA: Algorithm-Architecture Co-optimized Binary Transformer Accelerator for Edge Inference
by: Qiao, Ye, et al.
Published: (2025)

GPU Acceleration of Sparse Fully Homomorphic Encrypted DNNs
by: D'Agata, Lara, et al.
Published: (2026)

ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
by: Zeng, Chao, et al.
Published: (2024)

GraphNet: A Large-Scale Computational Graph Dataset for Tensor Compiler Research
by: Li, Xinqi, et al.
Published: (2025)

Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
by: Jiang, Wei, et al.
Published: (2024)

A Wireless Collaborated Inference Acceleration Framework for Plant Disease Recognition
by: Zhu, Hele, et al.
Published: (2025)

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
by: Jiang, Xuanlin, et al.
Published: (2024)

LCD: Advancing Extreme Low-Bit Clustering for Large Language Models via Knowledge Distillation
by: Liu, Fangxin, et al.
Published: (2025)

SparseMap: A Sparse Tensor Accelerator Framework Based on Evolution Strategy
by: Zhao, Boran, et al.
Published: (2025)

Spin: An Efficient Secure Computation Framework with GPU Acceleration
by: Jiang, Wuxuan, et al.
Published: (2024)

IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs
by: Mao, Yuzhen, et al.
Published: (2024)

Generating Full-field Evolution of Physical Dynamics from Irregular Sparse Observations
by: Chen, Panqi, et al.
Published: (2025)

From Sequential Nodes to GPU Batches: Parallel Branch and Bound for Optimal $k$-Sparse GLMs
by: Liu, Jiachang, et al.
Published: (2026)

Deep Symbolic Optimization for Combinatorial Optimization: Accelerating Node Selection by Discovering Potential Heuristics
by: Liu, Hongyu, et al.
Published: (2024)

Toward Robust and Efficient ML-Based GPU Caching for Modern Inference
by: Chen, Peng, et al.
Published: (2025)

KernelSkill: A Multi-Agent Framework for GPU Kernel Optimization
by: Sun, Qitong, et al.
Published: (2026)

Accelerating Mobile Inference through Fine-Grained CPU-GPU Co-Execution
by: Li, Zhuojin, et al.
Published: (2025)

Parallel Track Transformers: Enabling Fast GPU Inference with Reduced Synchronization
by: Wang, Chong, et al.
Published: (2026)