Cover Image

Short-Context Dominance: How Much Local Context Natural Language Actually Needs?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Vakilian, Vala, Wang, Zimeng, Rawat, Ankit Singh, Thrampoulidis, Christos
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models
by: Deng, Wenlong, et al.
Published: (2024)

Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations
by: Zhao, Yize, et al.
Published: (2024)

In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
by: Deora, Puneesh, et al.
Published: (2025)

Benchmarking LLMs for Pairwise Causal Discovery in Biomedical and Multi-Domain Contexts
by: Anuyah, Sydney, et al.
Published: (2026)

Layer-wise Positional Bias in Short-Context Language Modeling
by: Rahimi, Maryam, et al.
Published: (2026)

Natural Context Drift Undermines the Natural Language Understanding of Large Language Models
by: Wu, Yulong, et al.
Published: (2025)

Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?
by: Tan, Hexiang, et al.
Published: (2024)

A Statistical Framework for Data-dependent Retrieval-Augmented Models
by: Basu, Soumya, et al.
Published: (2024)

Task--Specificity Score: Measuring How Much Instructions Really Matter for Supervision
by: Kadasi, Pritam, et al.
Published: (2026)

When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models
by: Zheng, Yingming, et al.
Published: (2025)

No Clustering, No Routing: How Transformers Actually Process Rare Tokens
by: Liu, Jing
Published: (2025)

BertaQA: How Much Do Language Models Know About Local Culture?
by: Etxaniz, Julen, et al.
Published: (2024)

Interpreting User Requests in the Context of Natural Language Standing Instructions
by: Moghe, Nikita, et al.
Published: (2023)

Leveraging In-Context Learning for Language Model Agents
by: Gupta, Shivanshu, et al.
Published: (2025)

Context-Enriched Natural Language Descriptions of Vessel Trajectories
by: Patroumpas, Kostas, et al.
Published: (2026)

How Persuasive is Your Context?
by: Nguyen, Tu, et al.
Published: (2025)

How Much Do LLMs Hallucinate in Document Q&A Scenarios? A 172-Billion-Token Study Across Temperatures, Context Lengths, and Hardware Platforms
by: Roig, JV
Published: (2026)

Context-level Language Modeling by Learning Predictive Context Embeddings
by: Dai, Beiya, et al.
Published: (2025)

Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP
by: Goldman, Omer, et al.
Published: (2024)

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
by: Yen, Howard, et al.
Published: (2024)

How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment
by: Huang, Heyan, et al.
Published: (2024)

ContextGuard: Structured Self-Auditing for Context Learning in Language Models
by: Jin, Hongbo, et al.
Published: (2026)

Residual Context Diffusion Language Models
by: Hu, Yuezhou, et al.
Published: (2026)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
by: Li, Jiaqi, et al.
Published: (2023)

Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond
by: Li, Yingcong, et al.
Published: (2024)

EMA Is Not All You Need: Mapping the Boundary Between Structure and Content in Recurrent Context
by: Singh, Arth
Published: (2026)

How Much Can RAG Help the Reasoning of LLM?
by: Liu, Jingyu, et al.
Published: (2024)

Do Large Language Models Know How Much They Know?
by: Prato, Gabriele, et al.
Published: (2025)

Beyond Questions: Evaluating What Large Language Models (Actually) Know
by: Giordano, Luca, et al.
Published: (2026)

How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models
by: Kim, Minsung, et al.
Published: (2025)

Context Engineering 2.0: The Context of Context Engineering
by: Hua, Qishuo, et al.
Published: (2025)

s3: You Don't Need That Much Data to Train a Search Agent via RL
by: Jiang, Pengcheng, et al.
Published: (2025)

How Much Do LLMs Hallucinate across Languages? On Realistic Multilingual Estimation of LLM Hallucination
by: Islam, Saad Obaid ul, et al.
Published: (2025)

How do Language Models Bind Entities in Context?
by: Feng, Jiahai, et al.
Published: (2023)

Language Model Cascades: Token-level uncertainty and beyond
by: Gupta, Neha, et al.
Published: (2024)

Unlocking In-Context Learning for Natural Datasets Beyond Language Modelling
by: Bratulić, Jelena, et al.
Published: (2025)

Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering
by: Bigelow, Eric, et al.
Published: (2025)

Think before you speak: Training Language Models With Pause Tokens
by: Goyal, Sachin, et al.
Published: (2023)

Bridging Local Details and Global Context in Text-Attributed Graphs
by: Wang, Yaoke, et al.
Published: (2024)

Context-DPO: Aligning Language Models for Context-Faithfulness
by: Bi, Baolong, et al.
Published: (2024)