Imagen de Portada

Attention Sparsity is Input-Stable: Training-Free Sparse Attention for Video Generation via Offline Sparsity Profiling and Online QK Co-Clustering

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Luo, Jiayi, Chen, Jiayu, Wang, Jiankun, Wang, Cong, Zhu, Hanxin, Sun, Qingyun, Gao, Chen, Chen, Zhibo, Li, Jianxin
Formato:	Preprint
Publicado:	2026
Materias:	Computer Vision and Pattern Recognition
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation
por: Luo, Jiayi, et al.
Publicado: (2026)

Post-Training Sparse Attention with Double Sparsity
por: Yang, Shuo, et al.
Publicado: (2024)

Scaling Attention via Feature Sparsity
por: Xie, Yan, et al.
Publicado: (2026)

OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance
por: Wang, Cong, et al.
Publicado: (2026)

Sparse Recovery for Holographic MIMO Channels: Leveraging the Clustered Sparsity
por: Guo, Yuqing, et al.
Publicado: (2024)

STS: Efficient Sparse Attention with Speculative Token Sparsity
por: Xu, Ceyu, et al.
Publicado: (2026)

Rectified SpaAttn: Revisiting Attention Sparsity for Efficient Video Generation
por: Liu, Xuewen, et al.
Publicado: (2025)

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention
por: Zhang, Jintao, et al.
Publicado: (2025)

GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion
por: Zhu, Hanxin, et al.
Publicado: (2026)

UNIQUE: Universal Top-k Sparse Attention for Training-free Inference and Sparsity-aware Training
por: Deng, Keqi, et al.
Publicado: (2026)

Attention Condensation via Sparsity Induced Regularized Training
por: Sason, Eli, et al.
Publicado: (2025)

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
por: Qiu, Zihan, et al.
Publicado: (2025)

Input-Aware Sparse Attention for Real-Time Co-Speech Video Generation
por: Lu, Beijia, et al.
Publicado: (2025)

RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning
por: Hu, Junhao, et al.
Publicado: (2025)

Spark Transformer: Reactivating Sparsity in FFN and Attention
por: You, Chong, et al.
Publicado: (2025)

Dynamic Sparse Training with Structured Sparsity
por: Lasby, Mike, et al.
Publicado: (2023)

SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer
por: Fang, Tongcheng, et al.
Publicado: (2026)

BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformers
por: Xiao, Chaodong, et al.
Publicado: (2026)

BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
por: Yuan, Jiayi, et al.
Publicado: (2025)

HashAttention: Semantic Sparsity for Faster Inference
por: Desai, Aditya, et al.
Publicado: (2024)

FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion
por: Liu, Akide, et al.
Publicado: (2025)

S-HPLB: Efficient LLM Attention Serving via Sparsity-Aware Head Parallelism Load Balance
por: Liu, Di, et al.
Publicado: (2026)

Light Field Compression Based on Implicit Neural Representation
por: Wang, Henan, et al.
Publicado: (2024)

From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation
por: Ren, Yuxin, et al.
Publicado: (2026)

Compact Attention: Exploiting Structured Spatio-Temporal Sparsity for Fast Video Generation
por: Li, Qirui, et al.
Publicado: (2025)

Twilight: Adaptive Attention Sparsity with Hierarchical Top-$p$ Pruning
por: Lin, Chaofan, et al.
Publicado: (2025)

Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding
por: Fan, Wang, et al.
Publicado: (2026)

Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
por: Xi, Haocheng, et al.
Publicado: (2025)

Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity
por: Arnob, Samin Yeasar, et al.
Publicado: (2025)

SATA: Sparsity-Aware Scheduling for Selective Token Attention
por: Fan, Zhenkun, et al.
Publicado: (2026)

Crisp Attention: Regularizing Transformers via Structured Sparsity
por: Gandhi, Sagar, et al.
Publicado: (2025)

DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Training
por: Tan, Xin, et al.
Publicado: (2025)

Discrepancy Minimization in Input-Sparsity Time
por: Deng, Yichuan, et al.
Publicado: (2022)

Sparsity-Accelerated Training for Large Language Models
por: Ma, Da, et al.
Publicado: (2024)

Sparsity Forcing: Reinforcing Token Sparsity of MLLMs
por: Chen, Feng, et al.
Publicado: (2025)

LVSA: Training-Free Sparse Attention for Long Video Diffusion
por: Glorian, Gael, et al.
Publicado: (2026)

Embody4D: A Generalist 4D World Model for Embodied AI
por: Tu, Peiyan, et al.
Publicado: (2026)

Intrinsic and Extrinsic Organized Attention: Softmax Invariance and Network Sparsity
por: Fasina, Oluwadamilola, et al.
Publicado: (2025)

Multilayer Dataflow: Orchestrate Butterfly Sparsity to Accelerate Attention Computation
por: Wu, Haibin, et al.
Publicado: (2024)

CASP: Compression of Large Multimodal Models Based on Attention Sparsity
por: Gholami, Mohsen, et al.
Publicado: (2025)