Copertina

Overcoming Long-Context Limitations of State-Space Models via Context-Dependent Sparse Attention

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhan, Zhihao, Zhao, Jianan, Zhu, Zhaocheng, Tang, Jian
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Computation and Language I.2.7
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time
di: Zhao, Mingkuan, et al.
Pubblicazione: (2026)

CRISP: Persistent Concept Unlearning via Sparse Autoencoders
di: Ashuach, Tomer, et al.
Pubblicazione: (2025)

Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey
di: Huang, Yunpeng, et al.
Pubblicazione: (2023)

In-Context Algebra
di: Todd, Eric, et al.
Pubblicazione: (2025)

Structured-Sparse Attention for Entity Tracking with Subquadratic Sequence Complexity
di: Zhao, Hangyue, et al.
Pubblicazione: (2026)

Bayesian Attention Mechanism: A Probabilistic Framework for Positional Encoding and Context Length Extrapolation
di: Bianchessi, Arthur S., et al.
Pubblicazione: (2025)

LCFO: Long Context and Long Form Output Dataset and Benchmarking
di: Costa-jussà, Marta R., et al.
Pubblicazione: (2024)

SPHERICAL KV: Angle-Domain Attention and Rate-Distortion Retention for Efficient Long-Context Inference
di: Chauhan, Anay, et al.
Pubblicazione: (2026)

Efficient Solutions For An Intriguing Failure of LLMs: Long Context Window Does Not Mean LLMs Can Analyze Long Sequences Flawlessly
di: Hosseini, Peyman, et al.
Pubblicazione: (2024)

Context-Aware Clustering using Large Language Models
di: Tipirneni, Sindhu, et al.
Pubblicazione: (2024)

Comonadic Morphophonology: A Compositional Framework for Context-Dependent Morphological Rules in Finnish
di: Jang, Yongseok
Pubblicazione: (2026)

Surprise Calibration for Better In-Context Learning
di: Tan, Zhihang, et al.
Pubblicazione: (2025)

Recurrent Memory-Augmented Transformers with Chunked Attention for Long-Context Language Modeling
di: Kashyap, Ankit
Pubblicazione: (2025)

Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs
di: Cui, Wanyun, et al.
Pubblicazione: (2025)

QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm
di: Zhou, Qirui, et al.
Pubblicazione: (2025)

ICG: Improving Cover Image Generation via MLLM-based Prompting and Personalized Preference Alignment
di: Bian, Zhipeng, et al.
Pubblicazione: (2026)

Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs
di: Paulsen, Norman
Pubblicazione: (2025)

TIME: Temporally Intelligent Meta-reasoning Engine for Context-Triggered Explicit Reasoning
di: Das, Susmit
Pubblicazione: (2026)

Positional Failures in Long-Context LLMs: A Blind Spot in Reasoning Benchmarks
di: Zhang, Chuyifei, et al.
Pubblicazione: (2026)

KV-Fold: One-Step KV-Cache Recurrence for Long-Context Inference
di: Nadali, Alireza, et al.
Pubblicazione: (2026)

IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark for LLMs
di: Manikantan, Kawshik, et al.
Pubblicazione: (2024)

Overcoming the Generalization Limits of SLM Finetuning for Shape-Based Extraction of Datatype and Object Properties
di: Ringwald, Célian, et al.
Pubblicazione: (2025)

Hardware Co-Design Scaling Laws via Roofline Modelling for On-Device LLMs
di: Sun, Luoyang, et al.
Pubblicazione: (2026)

Enhancing In-Context Learning via Implicit Demonstration Augmentation
di: Zhou, Xiaoling, et al.
Pubblicazione: (2024)

Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts
di: Chourasia, Shivam, et al.
Pubblicazione: (2026)

Is Less More? Quality, Quantity and Context in Idiom Processing with Natural Language Models
di: Knietaite, Agne, et al.
Pubblicazione: (2024)

CG-TTRL: Context-Guided Test-Time Reinforcement Learning for On-Device Large Language Models
di: Hosseini, Peyman, et al.
Pubblicazione: (2025)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
di: Luo, Yuqi, et al.
Pubblicazione: (2024)

Context-Dependent Affordance Computation in Vision-Language Models
di: Farzulla, Murad
Pubblicazione: (2026)

Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text
di: Oketunji, Abiodun Finbarrs
Pubblicazione: (2023)

AsyncTLS: Efficient Generative LLM Inference with Asynchronous Two-level Sparse Attention
di: Hu, Yuxuan, et al.
Pubblicazione: (2026)

Scaling Synthetic Logical Reasoning Datasets with Context-Sensitive Declarative Grammars
di: Sileo, Damien
Pubblicazione: (2024)

Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders
di: Patel, Het, et al.
Pubblicazione: (2026)

BabyLlama-2: Ensemble-Distilled Models Consistently Outperform Teachers With Limited Data
di: Tastet, Jean-Loup, et al.
Pubblicazione: (2024)

LoRS: Efficient Low-Rank Adaptation for Sparse Large Language Model
di: Hu, Yuxuan, et al.
Pubblicazione: (2025)

Rethinking Addressing in Language Models via Contexualized Equivariant Positional Encoding
di: Zhu, Jiajun, et al.
Pubblicazione: (2025)

Lossless Prompt Compression via Dictionary-Encoding and In-Context Learning: Enabling Cost-Effective LLM Analysis of Repetitive Data
di: de Campos, Andresa Rodrigues, et al.
Pubblicazione: (2026)

Chain and Causal Attention for Efficient Entity Tracking
di: Fagnou, Erwan, et al.
Pubblicazione: (2024)

Detecting Data Contamination in LLMs via In-Context Learning
di: Zawalski, Michał, et al.
Pubblicazione: (2025)

Overcoming Black-box Attack Inefficiency with Hybrid and Dynamic Select Algorithms
di: Belde, Abhinay Shankar, et al.
Pubblicazione: (2025)