Copertina

MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Liu, Dong, Yu, Yanxuan, Lengerich, Ben, Wu, Ying Nian
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation
di: Liu, Dong, et al.
Pubblicazione: (2025)

AdaCorrection: Adaptive Offset Cache Correction for Accurate Diffusion Transformers
di: Liu, Dong, et al.
Pubblicazione: (2026)

PiKV: KV Cache Management System for Mixture of Experts
di: Liu, Dong, et al.
Pubblicazione: (2025)

CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference
di: Liu, Dong, et al.
Pubblicazione: (2025)

$π$-Attention: Periodic Sparse Transformers for Efficient Long-Context Modeling
di: Liu, Dong, et al.
Pubblicazione: (2025)

SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
di: Liu, Dong, et al.
Pubblicazione: (2025)

LOOKAT: Lookup-Optimized Key-Attention for Memory-Efficient Transformers
di: Karmore, Aryan
Pubblicazione: (2026)

Hamming Attention Distillation: Binarizing Keys and Queries for Efficient Long-Context Transformers
di: Horton, Mark, et al.
Pubblicazione: (2025)

HSGM: Hierarchical Segment-Graph Memory for Scalable Long-Text Semantics
di: Liu, Dong, et al.
Pubblicazione: (2025)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
di: Ling Team, et al.
Pubblicazione: (2025)

The Geometric Reasoner: Manifold-Informed Latent Foresight Search for Long-Context Reasoning
di: Zhuang, Ren, et al.
Pubblicazione: (2026)

CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
di: Zhao, Runsong, et al.
Pubblicazione: (2026)

Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning
di: Liu, Dong, et al.
Pubblicazione: (2026)

Designing Large Foundation Models for Efficient Training and Inference: A Survey
di: Liu, Dong, et al.
Pubblicazione: (2024)

SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
di: Gao, Yizhao, et al.
Pubblicazione: (2025)

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning
di: Zhang, Kehao, et al.
Pubblicazione: (2026)

COSMIR: Chain Orchestrated Structured Memory for Iterative Reasoning over Long Context
di: Gupta, Naman, et al.
Pubblicazione: (2025)

Hydra: A Modular Architecture for Efficient Long-Context Reasoning
di: Chaudhary, Siddharth, et al.
Pubblicazione: (2025)

SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training
di: Li, Zhouyang, et al.
Pubblicazione: (2025)

Efficient Low Rank Attention for Long-Context Inference in Large Language Models
di: Li, Tenghui, et al.
Pubblicazione: (2025)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
di: Xiao, Chaojun, et al.
Pubblicazione: (2024)

Efficient and Scalable Fine-Tune of Language Models for Genome Understanding
di: Zhan, Huixin, et al.
Pubblicazione: (2024)

MELODI: Exploring Memory Compression for Long Contexts
di: Chen, Yinpeng, et al.
Pubblicazione: (2024)

ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval
di: Yang, David H., et al.
Pubblicazione: (2026)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
di: MiniCPM Team, et al.
Pubblicazione: (2026)

Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
di: Ni, Wentao, et al.
Pubblicazione: (2026)

An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference
di: Yao, Feiyu, et al.
Pubblicazione: (2026)

RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning
di: Hu, Junhao, et al.
Pubblicazione: (2025)

MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference
di: Zhou, Ruijie, et al.
Pubblicazione: (2026)

Long-Context Attention Benchmark: From Kernel Efficiency to Distributed Context Parallelism
di: Bu, Tao, et al.
Pubblicazione: (2025)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
di: Chen, Yingfa, et al.
Pubblicazione: (2025)

Attention in Constant Time: Vashista Sparse Attention for Long-Context Decoding with Exponential Guarantees
di: Nobaub, Vashista
Pubblicazione: (2026)

Paged Attention Meets FlexAttention: Unlocking Long-Context Efficiency in Deployed Inference
di: Joshi, Thomas, et al.
Pubblicazione: (2025)

MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
di: Yu, Hongli, et al.
Pubblicazione: (2025)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
di: Chen, Yingfa, et al.
Pubblicazione: (2026)

Training Long-Context LLMs Efficiently via Chunk-wise Optimization
di: Li, Wenhao, et al.
Pubblicazione: (2025)

Evaluating Long-Context Reasoning in LLM-Based WebAgents
di: Chung, Andy, et al.
Pubblicazione: (2025)

MOM: Memory-Efficient Offloaded Mini-Sequence Inference for Long Context Language Models
di: Zhang, Junyang, et al.
Pubblicazione: (2025)

MoBA: Mixture of Block Attention for Long-Context LLMs
di: Lu, Enzhe, et al.
Pubblicazione: (2025)

LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
di: Wang, Guangtao, et al.
Pubblicazione: (2025)