Imagen de Portada

Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Wenhu, Wu, Yiming, Wang, Huanyu, Liu, Yaoyang, Dou, Huanzhang, Yang, Senqiao, Wu, Sitong, Zhao, Hanbin, Jia, Jiaya
Formato:	Preprint
Publicado:	2026
Materias:	Computer Vision and Pattern Recognition
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning
por: Yang, Senqiao, et al.
Publicado: (2025)

Does Your Vision-Language Model Get Lost in the Long Video Sampling Dilemma?
por: Qu, Tianyuan, et al.
Publicado: (2025)

Unified Language-driven Zero-shot Domain Adaptation
por: Yang, Senqiao, et al.
Publicado: (2024)

Logits-Based Finetuning
por: Li, Jingyao, et al.
Publicado: (2025)

LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model
por: Yang, Senqiao, et al.
Publicado: (2023)

VisionZip: Longer is Better but Not Necessary in Vision Language Models
por: Yang, Senqiao, et al.
Publicado: (2024)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
por: Chen, Yukang, et al.
Publicado: (2023)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
por: Lai, Xin, et al.
Publicado: (2024)

RoboCoder: Robotic Learning from Basic Skills to General Tasks with Large Language Models
por: Li, Jingyao, et al.
Publicado: (2024)

FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion
por: Chen, Zhuokun, et al.
Publicado: (2026)

Efficient Length-Generalizable Attention via Causal Retrieval for Long-Context Language Modeling
por: Hu, Xiang, et al.
Publicado: (2024)

SPLA: Block Sparse Plus Linear Attention for Long Context Modeling
por: Wang, Bailin, et al.
Publicado: (2026)

AB-Sparse: Sparse Attention with Adaptive Block Size for Accurate and Efficient Long-Context Inference
por: Liu, Di, et al.
Publicado: (2026)

GVDIFF: Grounded Text-to-Video Generation with Diffusion Models
por: Dou, Huanzhang, et al.
Publicado: (2024)

SearchGym: Bootstrapping Real-World Search Agents via Cost-Effective and High-Fidelity Environment Simulation
por: Zhang, Xichen, et al.
Publicado: (2026)

RRAttention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference
por: Liu, Siran, et al.
Publicado: (2026)

Overcoming Long-Context Limitations of State-Space Models via Context-Dependent Sparse Attention
por: Zhan, Zhihao, et al.
Publicado: (2025)

MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformers
por: Zhang, Yuechen, et al.
Publicado: (2025)

$π$-Attention: Periodic Sparse Transformers for Efficient Long-Context Modeling
por: Liu, Dong, et al.
Publicado: (2025)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
por: MiniCPM Team, et al.
Publicado: (2026)

Mass Concept Erasure in Diffusion Models with Concept Hierarchy
por: Tu, Jiahang, et al.
Publicado: (2026)

In-Context LoRA for Diffusion Transformers
por: Huang, Lianghua, et al.
Publicado: (2024)

SparseD: Sparse Attention for Diffusion Language Models
por: Wang, Zeqing, et al.
Publicado: (2025)

Efficient Low Rank Attention for Long-Context Inference in Large Language Models
por: Li, Tenghui, et al.
Publicado: (2025)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
por: You, Zeng, et al.
Publicado: (2025)

LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
por: Xi, Haocheng, et al.
Publicado: (2026)

Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention
por: Xiao, Emily, et al.
Publicado: (2025)

BFLA: Block-Filtered Long-Context Attention Mechanism
por: Wu, Chong, et al.
Publicado: (2026)

SemanticMIM: Marring Masked Image Modeling with Semantics Compression for General Visual Representation
por: Yuan, Yike, et al.
Publicado: (2024)

Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition
por: Zhong, Zhisheng, et al.
Publicado: (2024)

SmartSwitch: Advancing LLM Reasoning by Overcoming Underthinking via Promoting Deeper Thought Exploration
por: Zhang, Xichen, et al.
Publicado: (2025)

Modular Customization of Diffusion Models via Blockwise-Parameterized Low-Rank Adaptation
por: Zhu, Mingkang, et al.
Publicado: (2025)

Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
por: Yan, Siyuan, et al.
Publicado: (2025)

VMoBA: Mixture-of-Block Attention for Video Diffusion Models
por: Wu, Jianzong, et al.
Publicado: (2025)

LogoSticker: Inserting Logos into Diffusion Models for Customized Generation
por: Zhu, Mingkang, et al.
Publicado: (2024)

MTraining: Distributed Dynamic Sparse Attention for Efficient Ultra-Long Context Training
por: Li, Wenxuan, et al.
Publicado: (2025)

MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speech
por: Wang, Chengyao, et al.
Publicado: (2025)

Gated Sparse Attention: Combining Computational Efficiency with Training Stability for Long-Context Language Models
por: Shen, Alfred, et al.
Publicado: (2026)

Long-Context Generalization with Sparse Attention
por: Vasylenko, Pavlo, et al.
Publicado: (2025)

VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models
por: Wu, Tao, et al.
Publicado: (2024)