Imagen de Portada

HiCI: Hierarchical Construction-Integration for Long-Context Attention

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zeng, Xiangyu, Xu, Qi, Wang, Yunke, Xu, Chang
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
por: Zhu, Kan, et al.
Publicado: (2025)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
por: Chen, Yingfa, et al.
Publicado: (2025)

MoBA: Mixture of Block Attention for Long-Context LLMs
por: Lu, Enzhe, et al.
Publicado: (2025)

HiSpec: Hierarchical Speculative Decoding for LLMs
por: Kumar, Avinash, et al.
Publicado: (2025)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
por: Zhu, Qianchao, et al.
Publicado: (2024)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
por: Chen, Yingfa, et al.
Publicado: (2026)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
por: MiniCPM Team, et al.
Publicado: (2026)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
por: Li, Zeju, et al.
Publicado: (2026)

DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
por: Huang, Yuxiang, et al.
Publicado: (2026)

SEAL: Scaling to Emphasize Attention for Long-Context Retrieval
por: Lee, Changhun, et al.
Publicado: (2025)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
por: Zhu, Shiyi, et al.
Publicado: (2023)

HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA
por: Chen, Xinyue, et al.
Publicado: (2024)

HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation
por: Wu, Peilin, et al.
Publicado: (2025)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
por: Hsieh, Cheng-Yu, et al.
Publicado: (2024)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
por: Ling Team, et al.
Publicado: (2025)

HiFloat4 Format for Language Model Pre-training on Ascend NPUs
por: Taghian, Mehran, et al.
Publicado: (2026)

Sliding Window Attention Training for Efficient Large Language Models
por: Fu, Zichuan, et al.
Publicado: (2025)

Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack
por: Xu, Xiaoyue, et al.
Publicado: (2024)

100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?
por: Yang, Wang, et al.
Publicado: (2025)

From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models
por: Xu, Chejian, et al.
Publicado: (2025)

LycheeCluster: Efficient Long-Context Inference with Structure-Aware Chunking and Hierarchical KV Indexing
por: Li, Dongfang, et al.
Publicado: (2026)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
por: Xiao, Chaojun, et al.
Publicado: (2024)

LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
por: Wang, Guangtao, et al.
Publicado: (2025)

LongSafety: Enhance Safety for Long-Context LLMs
por: Huang, Mianqiu, et al.
Publicado: (2024)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
por: Leng, Jiaqi, et al.
Publicado: (2025)

Attention Needs to Focus: A Unified Perspective on Attention Allocation
por: Fu, Zichuan, et al.
Publicado: (2026)

Revisiting In-Context Learning with Long Context Language Models
por: Baek, Jinheon, et al.
Publicado: (2024)

Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning
por: Yang, Wang, et al.
Publicado: (2025)

Attention Smoothing Is All You Need For Unlearning
por: Zade, Saleh Zare, et al.
Publicado: (2026)

LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
por: Yang, Penghui, et al.
Publicado: (2025)

Decomposing Attention To Find Context-Sensitive Neurons
por: Gibson, Alex
Publicado: (2025)

Which Attention Heads Matter for In-Context Learning?
por: Yin, Kayo, et al.
Publicado: (2025)

Speculative Decoding Meets Quantization: Compatibility Evaluation and Hierarchical Framework Design
por: Zhang, Yudi, et al.
Publicado: (2025)

The Impossibility Triangle of Long-Context Modeling
por: Zhou, Yan
Publicado: (2026)

PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents
por: Gu, Zhuohan, et al.
Publicado: (2026)

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
por: Jin, Hongye, et al.
Publicado: (2024)

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models
por: Huang, Jie, et al.
Publicado: (2023)

InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
por: Zhao, Weilin, et al.
Publicado: (2025)

LLoCO: Learning Long Contexts Offline
por: Tan, Sijun, et al.
Publicado: (2024)

Accelerating Prefilling for Long-Context LLMs via Sparse Pattern Sharing
por: Peng, Dan, et al.
Publicado: (2025)