Cover Image

FlashFormer: Whole-Model Kernels for Efficient Low-Batch Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nrusimha, Aniruddha, Brandon, William, Mishra, Mayank, Shen, Yikang, Panda, Rameswar, Ragan-Kelley, Jonathan, Kim, Yoon
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
by: Brandon, William, et al.
Published: (2024)

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
by: Nrusimha, Aniruddha, et al.
Published: (2024)

Gated Linear Attention Transformers with Hardware-Efficient Training
by: Yang, Songlin, et al.
Published: (2023)

PaTH Attention: Position Encoding via Accumulating Householder Transformations
by: Yang, Songlin, et al.
Published: (2025)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
by: Pan, Bowen, et al.
Published: (2024)

Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding
by: Ankner, Zachary, et al.
Published: (2024)

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
by: Shen, Yikang, et al.
Published: (2024)

Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study
by: Tan, Shawn, et al.
Published: (2024)

Diversity Measurement and Subset Selection for Instruction Tuning Datasets
by: Wang, Peiqi, et al.
Published: (2024)

Distilling to Hybrid Attention Models via KL-Guided Layer Selection
by: Li, Yanhong, et al.
Published: (2025)

Structured Code Representations Enable Data-Efficient Adaptation of Code Language Models
by: Agarwal, Mayank, et al.
Published: (2024)

Towards Verifiable Text Generation with Symbolic References
by: Hennigen, Lucas Torroba, et al.
Published: (2023)

Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping
by: Zhang, Muru, et al.
Published: (2025)

Fast Matrix Multiplications for Lookup Table-Quantized LLMs
by: Guo, Han, et al.
Published: (2024)

API Pack: A Massive Multi-Programming Language Dataset for API Call Generation
by: Guo, Zhen, et al.
Published: (2024)

Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models
by: Wang, Xinyi, et al.
Published: (2025)

Scattered Mixture-of-Experts Implementation
by: Tan, Shawn, et al.
Published: (2024)

Data Engineering for Scaling Language Models to 128K Context
by: Fu, Yao, et al.
Published: (2024)

Guided Optimization for Image Processing Pipelines
by: Ikarashi, Yuka, et al.
Published: (2021)

Parallelizing Linear Transformers with the Delta Rule over Sequence Length
by: Yang, Songlin, et al.
Published: (2024)

Exo 2: Growing a Scheduling Language
by: Ikarashi, Yuka, et al.
Published: (2024)

Towards Efficient Visual-Language Alignment of the Q-Former for Visual Reasoning Tasks
by: Kim, Sungkyung, et al.
Published: (2024)

Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inference
by: Shen, Ke, et al.
Published: (2024)

LangNav: Language as a Perceptual Representation for Navigation
by: Pan, Bowen, et al.
Published: (2023)

FlashBack:Efficient Retrieval-Augmented Language Modeling for Long Context Inference
by: Liu, Runheng, et al.
Published: (2024)

RevMUX: Data Multiplexing with Reversible Adapters for Efficient LLM Batch Inference
by: Xu, Yige, et al.
Published: (2024)

PartialFormer: Modeling Part Instead of Whole for Machine Translation
by: Zheng, Tong, et al.
Published: (2023)

Efficient Continual Pre-training by Mitigating the Stability Gap
by: Guo, Yiduo, et al.
Published: (2024)

DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
by: Yao, Jinwei, et al.
Published: (2024)

Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation
by: Caren, Matthew, et al.
Published: (2024)

FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
by: Hu, Zhanqiu, et al.
Published: (2025)

When LLMs Stop Following Steps: A Diagnostic Study of Procedural Execution in Language Models
by: Panda, Sailesh, et al.
Published: (2026)

Modular GPU Programming with Typed Perspectives
by: Bansal, Manya, et al.
Published: (2025)

FlashSampling: Fast and Memory-Efficient Exact Sampling
by: Ruiz, Tomas, et al.
Published: (2026)

FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
by: Zadouri, Ted, et al.
Published: (2026)

Bonsai: Compiling Queries to Pruned Tree Traversals
by: Root, Alexander J, et al.
Published: (2025)

Char-mander Use mBackdoor! A Study of Cross-lingual Backdoor Attacks in Multilingual LLMs
by: Beniwal, Himanshu, et al.
Published: (2025)

FlashDecoding++: Faster Large Language Model Inference on GPUs
by: Hong, Ke, et al.
Published: (2023)

BatchGEMBA: Token-Efficient Machine Translation Evaluation with Batched Prompting and Prompt Compression
by: Larionov, Daniil, et al.
Published: (2025)

ChameleonLLM: Batch-Aware Dynamic Low-Rank Adaptation via Inference-Time Clusters
by: Yuksel, Kamer Ali, et al.
Published: (2025)