Cover Image

SparseD: Sparse Attention for Diffusion Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Zeqing, Fang, Gongfan, Ma, Xinyin, Yang, Xingyi, Wang, Xinchao
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

dKV-Cache: The Cache for Diffusion Language Models
by: Ma, Xinyin, et al.
Published: (2025)

Thinkless: LLM Learns When to Think
by: Fang, Gongfan, et al.
Published: (2025)

Efficient Reasoning Models: A Survey
by: Feng, Sicheng, et al.
Published: (2025)

Q-ARVD: Quantizing Autoregressive Video Diffusion Models
by: Tang, Siao, et al.
Published: (2026)

ConciseHint: Boosting Efficient Reasoning via Continuous Concise Hints during Generation
by: Tang, Siao, et al.
Published: (2025)

Remix-DiT: Mixing Diffusion Transformers for Multi-Expert Denoising
by: Fang, Gongfan, et al.
Published: (2024)

dParallel: Learnable Parallel Decoding for dLLMs
by: Chen, Zigeng, et al.
Published: (2025)

dMoE: dLLMs with Learnable Block Experts
by: Feng, Sicheng, et al.
Published: (2026)

Mix-Quant: Quantized Prefilling, Precise Decoding for Agentic LLMs
by: Lu, Haiquan, et al.
Published: (2026)

MixReasoning: Switching Modes to Think
by: Lu, Haiquan, et al.
Published: (2025)

CoT-Valve: Length-Compressible Chain-of-Thought Tuning
by: Ma, Xinyin, et al.
Published: (2025)

dVoting: Fast Voting for dLLMs
by: Feng, Sicheng, et al.
Published: (2026)

In-Video Instructions: Visual Signals as Generative Control
by: Fang, Gongfan, et al.
Published: (2025)

TinyFusion: Diffusion Transformers Learned Shallow
by: Fang, Gongfan, et al.
Published: (2024)

Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient
by: Chen, Zigeng, et al.
Published: (2024)

AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising
by: Chen, Zigeng, et al.
Published: (2024)

Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching
by: Ma, Xinyin, et al.
Published: (2024)

SlimSAM: 0.1% Data Makes Segment Anything Slim
by: Chen, Zigeng, et al.
Published: (2023)

Isomorphic Pruning for Vision Models
by: Fang, Gongfan, et al.
Published: (2024)

Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Models
by: Wan, Guangnian, et al.
Published: (2026)

LiteFocus: Accelerated Diffusion Inference for Long Audio Synthesis
by: Tan, Zhenxiong, et al.
Published: (2024)

Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
by: Yu, Runpeng, et al.
Published: (2025)

Invisible Safety Threat: Malicious Finetuning for LLM via Steganography
by: Wan, Guangnian, et al.
Published: (2026)

Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modeling
by: Kong, Hanyang, et al.
Published: (2025)

VeriThinker: Learning to Verify Makes Reasoning Model Efficient
by: Chen, Zigeng, et al.
Published: (2025)

Language Model as Visual Explainer
by: Yang, Xingyi, et al.
Published: (2024)

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
by: Wang, Hongyu, et al.
Published: (2024)

Introducing Visual Perception Token into Multimodal Large Language Model
by: Yu, Runpeng, et al.
Published: (2025)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
by: Fang, Gongfan, et al.
Published: (2024)

DMax: Aggressive Parallel Decoding for dLLMs
by: Chen, Zigeng, et al.
Published: (2026)

Rectified Sparse Attention
by: Sun, Yutao, et al.
Published: (2025)

Punctuation-aware Hybrid Trainable Sparse Attention for Large Language Models
by: Qiu, Junxiang, et al.
Published: (2026)

SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
by: Shen, Zhenyi, et al.
Published: (2025)

Interpretable LLM Guardrails via Sparse Representation Steering
by: He, Zeqing, et al.
Published: (2025)

FASA: Frequency-aware Sparse Attention
by: Wang, Yifei, et al.
Published: (2026)

Hash3D: Training-free Acceleration for 3D Generation
by: Yang, Xingyi, et al.
Published: (2024)

Relation Rectification in Diffusion Model
by: Wu, Yinwei, et al.
Published: (2024)

LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
by: Xi, Haocheng, et al.
Published: (2026)

SPLA: Block Sparse Plus Linear Attention for Long Context Modeling
by: Wang, Bailin, et al.
Published: (2026)

Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping
by: Chen, Yao, et al.
Published: (2026)