Cover Image

Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Yuerong, Liu, Xiaoran, Li, Ruixiao, Liu, Zhigeng, Huang, Zengfeng, Guo, Qipeng, He, Ziwei, Qiu, Xipeng
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
by: Liu, Xiaoran, et al.
Published: (2025)

Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache
by: Liu, Xiaoran, et al.
Published: (2025)

Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
by: Liu, Xiaoran, et al.
Published: (2025)

ReAttention: Training-Free Infinite Context with Finite Attention Scope
by: Liu, Xiaoran, et al.
Published: (2024)

Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference
by: Huang, Jianuo, et al.
Published: (2025)

Thus Spake Long-Context Large Language Model
by: Liu, Xiaoran, et al.
Published: (2025)

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
by: Liu, Zhiyuan, et al.
Published: (2025)

FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation
by: He, Siyang, et al.
Published: (2026)

Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
by: Wu, Chengyue, et al.
Published: (2025)

Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
by: Xiao, Zhongyu, et al.
Published: (2026)

dLLM: Simple Diffusion Language Modeling
by: Zhou, Zhanhui, et al.
Published: (2026)

Fast-dLLM v2: Efficient Block-Diffusion LLM
by: Wu, Chengyue, et al.
Published: (2025)

DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels
by: Xu, Zhe, et al.
Published: (2024)

Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing
by: Long, Lingkun, et al.
Published: (2026)

LongWanjuan: Towards Systematic Measurement for Long Text Quality
by: Lv, Kai, et al.
Published: (2024)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
by: Peng, Runyu, et al.
Published: (2026)

DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting
by: Lv, Kai, et al.
Published: (2025)

Taming the Fragility of KV Cache Eviction in LLM Inference
by: Feng, Yuan, et al.
Published: (2025)

d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
by: Jiang, Yuchu, et al.
Published: (2025)

$R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction
by: Du, Zhenbang, et al.
Published: (2026)

Prism: Spectral-Aware Block-Sparse Attention
by: Wang, Xinghao, et al.
Published: (2026)

LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding
by: Xu, Chenkai, et al.
Published: (2025)

IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
by: Yang, Xintong, et al.
Published: (2026)

In-context KV-Cache Eviction for LLMs via Attention-Gate
by: Zeng, Zihao, et al.
Published: (2024)

NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
by: Chen, Yilong, et al.
Published: (2024)

DiRL: An Efficient Post-Training Framework for Diffusion Language Models
by: Zhu, Ying, et al.
Published: (2025)

LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
by: Zhou, Enshuai, et al.
Published: (2026)

LongSafety: Enhance Safety for Long-Context LLMs
by: Huang, Mianqiu, et al.
Published: (2024)

Reformulating KV Cache Eviction Problem for Long-Context LLM Inference
by: Mai, Tho, et al.
Published: (2026)

Full Parameter Fine-tuning for Large Language Models with Limited Resources
by: Lv, Kai, et al.
Published: (2023)

SABlock: Semantic-Aware KV Cache Eviction with Adaptive Compression Block Size
by: Chen, Jinhan, et al.
Published: (2025)

Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Query
by: Wang, Yixuan, et al.
Published: (2025)

Code Needs Comments: Enhancing Code LLMs with Comment Augmentation
by: Song, Demin, et al.
Published: (2024)

AdaLomo: Low-memory Optimization with Adaptive Learning Rate
by: Lv, Kai, et al.
Published: (2023)

CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective
by: Feng, Yuan, et al.
Published: (2025)

CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Eviction
by: Goel, Raghavv, et al.
Published: (2025)

CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences
by: Qin, Ziran, et al.
Published: (2025)

Can Language Models Learn to Skip Steps?
by: Liu, Tengxiao, et al.
Published: (2024)

Judge Q: Trainable Queries for Optimized Information Retention in KV Cache Eviction
by: Liu, Yijun, et al.
Published: (2025)

Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
by: He, Zhengfu, et al.
Published: (2024)