Cover Image

Evaluating Language Model Context Windows: A "Working Memory" Test and Inference-time Correction

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dsouza, Amanda, Glaze, Christopher, Shin, Changho, Sala, Frederic
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Benchmarking Agents in Insurance Underwriting Environments
by: Dsouza, Amanda, et al.
Published: (2026)

Weak-to-Strong Generalization Through the Data-Centric Lens
by: Shin, Changho, et al.
Published: (2024)

Empowering Working Memory for Large Language Model Agents
by: Guo, Jing, et al.
Published: (2023)

PSC: Extending Context Window of Large Language Models via Phase Shift Calibration
by: Zhu, Wenqiao, et al.
Published: (2025)

SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks
by: Orlanski, Gabriel, et al.
Published: (2026)

Unlocking the Working Memory of Large Language Models for Latent Reasoning
by: Aichberger, Lukas, et al.
Published: (2026)

MOM: Memory-Efficient Offloaded Mini-Sequence Inference for Long Context Language Models
by: Zhang, Junyang, et al.
Published: (2025)

Towards Inference-time Category-wise Safety Steering for Large Language Models
by: Bhattacharjee, Amrita, et al.
Published: (2024)

LongCodeBench: Evaluating Coding LLMs at 1M Context Windows
by: Rando, Stefano, et al.
Published: (2025)

InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformation
by: Cao, Bowen, et al.
Published: (2025)

Zero-Shot Robustification of Zero-Shot Models
by: Adila, Dyah, et al.
Published: (2023)

YaRN: Efficient Context Window Extension of Large Language Models
by: Peng, Bowen, et al.
Published: (2023)

Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
by: Huang, Chensen, et al.
Published: (2024)

Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness
by: Wang, Wenxuan
Published: (2024)

When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs
by: Khairi, Ammar, et al.
Published: (2025)

When Context Leads but Parametric Memory Follows in Large Language Models
by: Tao, Yufei, et al.
Published: (2024)

Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models
by: Sakarvadia, Mansi, et al.
Published: (2023)

Luna-2: Scalable Single-Token Evaluation with Small Language Models
by: Goel, Vatsal, et al.
Published: (2026)

Knowledge-Aware Self-Correction in Language Models via Structured Memory Graphs
by: Saha, Swayamjit
Published: (2025)

Sleep-time Compute: Beyond Inference Scaling at Test-time
by: Lin, Kevin, et al.
Published: (2025)

Window-based Membership Inference Attacks Against Fine-tuned Large Language Models
by: Chen, Yuetian, et al.
Published: (2026)

Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models
by: Xu, Ningyu, et al.
Published: (2026)

MeVe: A Modular System for Memory Verification and Effective Context Control in Language Models
by: Ottem, Andreas
Published: (2025)

Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style
by: Li, Yuepei, et al.
Published: (2024)

Gradual Forgetting: Logarithmic Compression for Extending Transformer Context Windows
by: Dickson, Billy, et al.
Published: (2025)

Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
by: Wei, Tianxin, et al.
Published: (2025)

Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently
by: Misra, Kanishka, et al.
Published: (2024)

DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration
by: Zhang, Hanzhi, et al.
Published: (2025)

IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
by: Yang, Xintong, et al.
Published: (2026)

A Comprehensive Evaluation of Large Language Models on Mental Illnesses in Arabic Context
by: Zahran, Noureldin, et al.
Published: (2025)

Specification Self-Correction: Mitigating In-Context Reward Hacking Through Test-Time Refinement
by: Gallego, Víctor
Published: (2025)

Adaptive Focus Memory for Language Models
by: Cruz, Christopher
Published: (2025)

Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities
by: Bertsch, Amanda, et al.
Published: (2025)

Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inference
by: Shen, Ke, et al.
Published: (2024)

Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models
by: Chen, Yushuo, et al.
Published: (2024)

Evaluation of Language Models in the Medical Context Under Resource-Constrained Settings
by: Posada, Andrea, et al.
Published: (2024)

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
by: Yen, Howard, et al.
Published: (2024)

LongSafety: Evaluating Long-Context Safety of Large Language Models
by: Lu, Yida, et al.
Published: (2025)

Is Free Self-Alignment Possible?
by: Adila, Dyah, et al.
Published: (2024)

Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
by: Warner, Benjamin, et al.
Published: (2024)