Cover Image

Sparser Block-Sparse Attention via Token Permutation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Xinghao, Wang, Pengyu, Zhang, Dong, Tan, Chenkun, Zhou, Shaojun, Liu, Zhaoxiang, Lian, Shiguo, Liu, Fangxu, Song, Kai, Qiu, Xipeng
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Prism: Spectral-Aware Block-Sparse Attention
by: Wang, Xinghao, et al.
Published: (2026)

Decoupled Proxy Alignment: Mitigating Language Prior Conflict for Multimodal Alignment in MLLM
by: Tan, Chenkun, et al.
Published: (2025)

UnifiedVisual: A Framework for Constructing Unified Vision-Language Datasets
by: Wang, Pengyu, et al.
Published: (2025)

InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
by: Wang, Pengyu, et al.
Published: (2024)

MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time
by: Zhang, Mozhi, et al.
Published: (2024)

BitStack: Any-Size Compression of Large Language Models in Variable Memory Environments
by: Wang, Xinghao, et al.
Published: (2024)

TP3M: Transformer-based Pseudo 3D Image Matching with Reference Image
by: Han, Liming, et al.
Published: (2024)

Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
by: Liu, Xiaoran, et al.
Published: (2025)

Patch-wise Auto-Encoder for Visual Anomaly Detection
by: Cui, Yajie, et al.
Published: (2023)

KAConvNet: Kolmogorov-Arnold Convolutional Networks for Vision Recognition
by: Liu, Zhaoxiang, et al.
Published: (2026)

A Large Vision-Language Model based Environment Perception System for Visually Impaired People
by: Chen, Zezhou, et al.
Published: (2025)

DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning
by: Wang, Xinghao, et al.
Published: (2024)

Sparser2Sparse: Single-shot Sparser-to-Sparse Learning for Spatial Transcriptomics Imputation with Natural Image Co-learning
by: Fang, Yaoyu, et al.
Published: (2025)

SLearnLLM: A Self-Learning Framework for Efficient Domain-Specific Adaptation of Large Language Models
by: Liu, Xiang, et al.
Published: (2025)

The Open-World Lottery Ticket Hypothesis for OOD Intent Classification
by: Zhou, Yunhua, et al.
Published: (2022)

LongSafety: Enhance Safety for Long-Context LLMs
by: Huang, Mianqiu, et al.
Published: (2024)

A Multimodal Benchmark Dataset and Model for Crop Disease Diagnosis
by: Liu, Xiang, et al.
Published: (2025)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
by: Lou, Chao, et al.
Published: (2024)

CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Models
by: Zhang, Wenjing, et al.
Published: (2024)

LeMiCa: Lexicographic Minimax Path Caching for Efficient Diffusion-Based Video Generation
by: Gao, Huanlin, et al.
Published: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
by: Ma, Zhicheng, et al.
Published: (2026)

What is the best model? Application-driven Evaluation for Large Language Models
by: Lian, Shiguo, et al.
Published: (2024)

PSTF-AttControl: Per-Subject-Tuning-Free Personalized Image Generation with Controllable Face Attributes
by: liu, Xiang, et al.
Published: (2025)

SpeechAgents: Human-Communication Simulation with Multi-Modal Multi-Agent Systems
by: Zhang, Dong, et al.
Published: (2024)

Optimizing for the Shortest Path in Denoising Diffusion Model
by: Chen, Ping, et al.
Published: (2025)

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
by: Wang, Junxuan, et al.
Published: (2025)

Hierarchical Deep Fusion Framework for Multi-dimensional Facial Forgery Detection -- The 2024 Global Deepfake Image Detection Challenge
by: Wang, Kohou, et al.
Published: (2025)

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
by: Zhang, Xin, et al.
Published: (2023)

Exploring Multi-Temperature Strategies for Token- and Rollout-Level Control in RLVR
by: Zhuang, Haomin, et al.
Published: (2025)

Beyond Attention Magnitude: Leveraging Inter-layer Rank Consistency for Efficient Vision-Language-Action Models
by: Liu, Peiju, et al.
Published: (2026)

XAttention: Block Sparse Attention with Antidiagonal Scoring
by: Xu, Ruyi, et al.
Published: (2025)

Chain-of-Trajectories: Unlocking the Intrinsic Generative Optimality of Diffusion Models via Graph-Theoretic Planning
by: Chen, Ping, et al.
Published: (2026)

RoomCraft: Controllable and Complete 3D Indoor Scene Generation
by: Zhou, Mengqi, et al.
Published: (2025)

Boosting Text-to-Image Diffusion Models via Core Token Attention-Based Seed Selection
by: Zhang, Yunzhe, et al.
Published: (2026)

Fuzzy Reasoning Chain (FRC): An Innovative Reasoning Framework from Fuzziness to Clarity
by: Chen, Ping, et al.
Published: (2025)

Methodology of Adapting Large English Language Models for Specific Cultural Contexts
by: Zhang, Wenjing, et al.
Published: (2024)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
by: Peng, Runyu, et al.
Published: (2026)

Beyond Geometry: Artistic Disparity Synthesis for Immersive 2D-to-3D
by: Chen, Ping, et al.
Published: (2026)

Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation
by: Yang, Shuo, et al.
Published: (2025)

ConvSearch-R1: Enhancing Query Reformulation for Conversational Search with Reasoning via Reinforcement Learning
by: Zhu, Changtai, et al.
Published: (2025)