Image de couverture de livre

Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Hu, Xiang, Leng, Jiaqi, Zhao, Jun, Tu, Kewei, Wu, Wei
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Efficient Length-Generalizable Attention via Causal Retrieval for Long-Context Language Modeling
par: Hu, Xiang, et autres
Publié: (2024)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
par: Leng, Jiaqi, et autres
Publié: (2025)

Augmenting Transformers with Recursively Composed Multi-grained Representations
par: Hu, Xiang, et autres
Publié: (2023)

A Systematic Study of Compositional Syntactic Transformer Language Models
par: Zhao, Yida, et autres
Publié: (2025)

Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale
par: Hu, Xiang, et autres
Publié: (2024)

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
par: Yuan, Jingyang, et autres
Publié: (2025)

MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokens
par: Chen, Yu, et autres
Publié: (2026)

Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models
par: Zhao, Yida, et autres
Publié: (2024)

$π$-Attention: Periodic Sparse Transformers for Efficient Long-Context Modeling
par: Liu, Dong, et autres
Publié: (2025)

Long-Context Generalization with Sparse Attention
par: Vasylenko, Pavlo, et autres
Publié: (2025)

Flash Multi-Head Feed-Forward Network
par: Zhang, Minshen, et autres
Publié: (2025)

DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration
par: Zhang, Hanzhi, et autres
Publié: (2025)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
par: MiniCPM Team, et autres
Publié: (2026)

DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
par: Huang, Yuxiang, et autres
Publié: (2026)

Agentic Recommender System with Hierarchical Belief-State Memory
par: Shen, Xiang, et autres
Publié: (2026)

LongHeads: Multi-Head Attention is Secretly a Long Context Processor
par: Lu, Yi, et autres
Publié: (2024)

In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
par: Tan, Zhen, et autres
Publié: (2025)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
par: Lou, Chao, et autres
Publié: (2024)

Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents
par: Sun, Haoran, et autres
Publié: (2025)

Efficient Context Scaling with LongCat ZigZag Attention
par: Zhang, Chen, et autres
Publié: (2025)

AllMem: A Memory-centric Recipe for Efficient Long-context Modeling
par: Wang, Ziming, et autres
Publié: (2026)

Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention
par: Weng, Yuzhe, et autres
Publié: (2024)

LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
par: Yang, Shang, et autres
Publié: (2025)

TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents
par: Li, Kai, et autres
Publié: (2026)

Long-term Task-oriented Agent: Proactive Long-term Intent Maintenance in Dynamic Environments
par: Shi, Qinglong, et autres
Publié: (2026)

Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
par: Hu, Junhao, et autres
Publié: (2026)

How Sparse Attention Approximates Exact Attention? Your Attention is Naturally $n^C$-Sparse
par: Deng, Yichuan, et autres
Publié: (2024)

SWAA: Sliding Window Attention Adaptation for Efficient and Quality Preserving Long Context Processing
par: Yu, Yijiong, et autres
Publié: (2025)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
par: Zhu, Kan, et autres
Publié: (2025)

SA-DiffuSeq: Addressing Computational and Scalability Challenges in Long-Document Generation with Sparse Attention
par: Christoforos, Alexandros, et autres
Publié: (2025)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
par: Chen, Yingfa, et autres
Publié: (2025)

Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation
par: Ai, Chaoyi, et autres
Publié: (2024)

HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
par: Gao, Yizhao, et autres
Publié: (2026)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
par: Chen, Yingfa, et autres
Publié: (2026)

HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model
par: Hu, Mengkang, et autres
Publié: (2024)

Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory
par: Hu, Sen, et autres
Publié: (2026)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
par: Ling Team, et autres
Publié: (2025)

InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
par: Zhao, Weilin, et autres
Publié: (2025)

HiCI: Hierarchical Construction-Integration for Long-Context Attention
par: Zeng, Xiangyu, et autres
Publié: (2026)

RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging
par: Wang, Bowen, et autres
Publié: (2025)