Copertina

Ragged Paged Attention: A High-Performance and Flexible LLM Inference Kernel for TPU

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Jiang, Jevin, Chen, Ying, Hechtman, Blake A., Zhang, Fenghui, Mu, Yarong
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Performance Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Flex-TPU: A Flexible TPU with Runtime Reconfigurable Dataflow Architecture
di: Elbtity, Mohammed, et al.
Pubblicazione: (2024)

Enabling Performant and Flexible Model-Internal Observability for LLM Inference
di: Yu, Nengneng, et al.
Pubblicazione: (2026)

Online Pseudo-average Shifting Attention(PASA) for Robust Low-precision LLM Inference: Algorithms and Numerical Analysis
di: Cheng, Long, et al.
Pubblicazione: (2025)

An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference
di: Yao, Feiyu, et al.
Pubblicazione: (2026)

Pushing the Envelope of LLM Inference on AI-PC and Intel GPUs
di: Georganas, Evangelos, et al.
Pubblicazione: (2025)

GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimization
di: Andrews, Martin, et al.
Pubblicazione: (2025)

Forecasting LLM Inference Performance via Hardware-Agnostic Analytical Modeling
di: Patwari, Rajeev, et al.
Pubblicazione: (2025)

ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inference
di: Yin, Wangsong, et al.
Pubblicazione: (2025)

FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
di: Shao, Zishan, et al.
Pubblicazione: (2025)

Private LLM Inference on Consumer Blackwell GPUs: A Practical Guide for Cost-Effective Local Deployment in SMEs
di: Knoop, Jonathan, et al.
Pubblicazione: (2026)

LLM Swiss Round: Aggregating Multi-Benchmark Performance via Competitive Swiss-System Dynamics
di: Liu, Jiashuo, et al.
Pubblicazione: (2025)

KernelBench: Can LLMs Write Efficient GPU Kernels?
di: Ouyang, Anne, et al.
Pubblicazione: (2025)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
di: Chu, Kexin, et al.
Pubblicazione: (2025)

On the Sustainability of AI Inferences in the Edge
di: Sobhani, Ghazal, et al.
Pubblicazione: (2025)

Research on Low-Latency Inference and Training Efficiency Optimization for Graph Neural Network and Large Language Model-Based Recommendation Systems
di: Zhao, Yushang, et al.
Pubblicazione: (2025)

SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference
di: Zhang, Jintao, et al.
Pubblicazione: (2025)

Neuralink: Fast LLM Inference on Smartphones with Neuron Co-Activation Linking
di: Wang, Tuowei, et al.
Pubblicazione: (2024)

Rapid Augmentations for Time Series (RATS): A High-Performance Library for Time Series Augmentation
di: Skaf, Wadie, et al.
Pubblicazione: (2026)

FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
di: Wu, Wenhao, et al.
Pubblicazione: (2026)

Bench360: Benchmarking Local LLM Inference from 360 Degrees
di: Stuhlmann, Linus, et al.
Pubblicazione: (2025)

ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression
di: Liu, Guangda, et al.
Pubblicazione: (2024)

FlashOmni: A Unified Sparse Attention Engine for Diffusion Transformers
di: Qiao, Liang, et al.
Pubblicazione: (2025)

Energy-Efficient Transformer Inference: Optimization Strategies for Time Series Classification
di: Kermani, Arshia, et al.
Pubblicazione: (2025)

SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training
di: Zhang, Jintao, et al.
Pubblicazione: (2025)

EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training
di: Yi, Qingao, et al.
Pubblicazione: (2025)

APOLLO: SGD-like Memory, AdamW-level Performance
di: Zhu, Hanqing, et al.
Pubblicazione: (2024)

ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
di: Zhao, Youpeng, et al.
Pubblicazione: (2024)

Reducing Latency of LLM Search Agent via Speculation-based Algorithm-System Co-Design
di: Huang, Zixiao, et al.
Pubblicazione: (2025)

Performance Modeling of Data Storage Systems using Generative Models
di: Al-Maeeni, Abdalaziz Rashid, et al.
Pubblicazione: (2023)

Kevin: Multi-Turn RL for Generating CUDA Kernels
di: Baronio, Carlo, et al.
Pubblicazione: (2025)

SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression
di: Mozaffari, Mohammad, et al.
Pubblicazione: (2024)

GreedySnake: Accelerating SSD-Offloaded LLM Training with Efficient Scheduling and Optimizer Step Overlapping
di: Yin, Yishu, et al.
Pubblicazione: (2025)

CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe
di: Saba, Tara, et al.
Pubblicazione: (2026)

KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta
di: Liao, Gang, et al.
Pubblicazione: (2025)

Ensuring Reliability of Curated EHR-Derived Data: The Validation of Accuracy for LLM/ML-Extracted Information and Data (VALID) Framework
di: Estevez, Melissa, et al.
Pubblicazione: (2025)

Quantum Neural Networks for Wind Energy Forecasting: A Comparative Study of Performance and Scalability with Classical Models
di: Hangun, Batuhan, et al.
Pubblicazione: (2025)

Model Compression and Efficient Inference for Large Language Models: A Survey
di: Wang, Wenxiao, et al.
Pubblicazione: (2024)

GPU Kernel Optimization Beyond Full Builds: An LLM Framework with Minimal Executable Programs
di: Chu, Ruifan, et al.
Pubblicazione: (2025)

KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels
di: Wang, Han, et al.
Pubblicazione: (2026)

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
di: Zheng, Wenhao, et al.
Pubblicazione: (2025)