Cover Image

IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mao, Yuzhen, Ester, Martin, Li, Ke
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

IceCache: Memory-efficient KV-cache Management for Long-Sequence LLMs
by: Mao, Yuzhen, et al.
Published: (2026)

TwinFormer: A Dual-Level Transformer for Long-Sequence Time-Series Forecasting
by: Kumavat, Mahima, et al.
Published: (2025)

Fast and Compact Tsetlin Machine Inference on CPUs Using Instruction-Level Optimization
by: Zeng, Yefan, et al.
Published: (2025)

FairyFuse: Multiplication-Free LLM Inference on CPUs via Fused Ternary Kernels
by: Zuo, Fei, et al.
Published: (2026)

YuriiFormer: A Suite of Nesterov-Accelerated Transformers
by: Zimin, Aleksandr, et al.
Published: (2026)

Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference
by: Zhang, Haolin, et al.
Published: (2025)

MatFormer: Nested Transformer for Elastic Inference
by: Devvrit, et al.
Published: (2023)

Accelerating Sparse Transformer Inference on GPU
by: Dai, Wenhao, et al.
Published: (2025)

SparAMX: Accelerating Compressed LLMs Token Generation on AMX-powered CPUs
by: AbouElhamayed, Ahmed F., et al.
Published: (2025)

CoFormer: Collaborating with Heterogeneous Edge Devices for Scalable Transformer Inference
by: Xu, Guanyu, et al.
Published: (2025)

Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Training
by: Luo, Cheng, et al.
Published: (2024)

ScalableHD: Scalable and High-Throughput Hyperdimensional Computing Inference on Multi-Core CPUs
by: Parikh, Dhruv, et al.
Published: (2025)

TeamFormer: Shallow Parallel Transformers with Progressive Approximation
by: Wang, Wei, et al.
Published: (2025)

Highly Optimized Kernels and Fine-Grained Codebooks for LLM Inference on Arm CPUs
by: Gope, Dibakar, et al.
Published: (2024)

MechaFormer: Sequence Learning for Kinematic Mechanism Design Automation
by: Bolanos, Diana, et al.
Published: (2025)

SFi-Former: Sparse Flow Induced Attention for Graph Transformer
by: Li, Zhonghao, et al.
Published: (2025)

From TLinFormer to TConstFormer: The Leap to Constant-Time Transformer Attention: Achieving O(1) Computation and O(1) KV Cache during Autoregressive Inference
by: Tang, Zhongpan
Published: (2025)

NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attention
by: Zhang, Tianyi, et al.
Published: (2024)

Forget, Then Recall: Learnable Compression and Selective Unfolding via Gist Sparse Attention
by: Mao, Yuzhen, et al.
Published: (2026)

GeoFormer: A Vision and Sequence Transformer-based Approach for Greenhouse Gas Monitoring
by: Khirwar, Madhav, et al.
Published: (2024)

CausalFormer: An Interpretable Transformer for Temporal Causal Discovery
by: Kong, Lingbai, et al.
Published: (2024)

Flash Inference: Near Linear Time Inference for Long Convolution Sequence Models and Beyond
by: Oncescu, Costin-Andrei, et al.
Published: (2024)

SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPC
by: Luo, Jinglong, et al.
Published: (2024)

ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector Attention
by: Shao, Jintian, et al.
Published: (2025)

Context-Former: Stitching via Latent Conditioned Sequence Modeling
by: Zhang, Ziqi, et al.
Published: (2024)

WaveFormer: Wavelet Embedding Transformer for Biomedical Signals
by: Irani, Habib, et al.
Published: (2026)

SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformers
by: Liu, Joseph, et al.
Published: (2024)

TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformer
by: Li, Jiayu, et al.
Published: (2025)

Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity
by: Li, Victor, et al.
Published: (2025)

DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging
by: Pagliardini, Matteo, et al.
Published: (2024)

TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
by: Wang, Haiyang, et al.
Published: (2024)

On the Resurgence of Recurrent Models for Long Sequences -- Survey and Research Opportunities in the Transformer Era
by: Tiezzi, Matteo, et al.
Published: (2024)

LongVQ: Long Sequence Modeling with Vector Quantization on Structured Memory
by: Liu, Zicheng, et al.
Published: (2024)

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
by: Sun, Hanshi, et al.
Published: (2024)

TruncFormer: Private LLM Inference Using Only Truncations
by: Yubeaton, Patrick, et al.
Published: (2024)

ASTRA: Communication-Efficient Acceleration for Multi-Device Transformer Inference
by: Liu, Xiao, et al.
Published: (2025)

Star Attention: Efficient LLM Inference over Long Sequences
by: Acharya, Shantanu, et al.
Published: (2024)

Neuromorphic Wireless Device-Edge Co-Inference via the Directed Information Bottleneck
by: Ke, Yuzhen, et al.
Published: (2024)

AlgoFormer: An Efficient Transformer Framework with Algorithmic Structures
by: Gao, Yihang, et al.
Published: (2024)

Peri-midFormer: Periodic Pyramid Transformer for Time Series Analysis
by: Wu, Qiang, et al.
Published: (2024)