Cover Image

Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Joo, Taejong, Klabjan, Diego
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

IW-GAE: Importance Weighted Group Accuracy Estimation for Improved Calibration and Model Selection in Unsupervised Domain Adaptation
by: Joo, Taejong, et al.
Published: (2023)

Improving self-training under distribution shifts via anchored confidence with theoretical guarantees
by: Joo, Taejong, et al.
Published: (2024)

Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration
by: Joo, Taejong, et al.
Published: (2025)

Regret Bounds and Reinforcement Learning Exploration of EXP-based Algorithms
by: Xu, Mengfan, et al.
Published: (2020)

Long-Context Attention Benchmark: From Kernel Efficiency to Distributed Context Parallelism
by: Bu, Tao, et al.
Published: (2025)

On Surprising Effectiveness of Masking Updates in Adaptive Optimizers
by: Joo, Taejong, et al.
Published: (2026)

Communication-Efficient Federated Low-Rank Update Algorithm and its Connection to Implicit Regularization
by: Park, Haemin, et al.
Published: (2024)

Revisiting In-Context Learning with Long Context Language Models
by: Baek, Jinheon, et al.
Published: (2024)

Long Context In-Context Compression by Getting to the Gist of Gisting
by: Petrov, Aleksandar, et al.
Published: (2025)

Multi-Layer Attention-Based Explainability via Transformers for Tabular Data
by: Gavito, Andrea Treviño, et al.
Published: (2023)

Paged Attention Meets FlexAttention: Unlocking Long-Context Efficiency in Deployed Inference
by: Joshi, Thomas, et al.
Published: (2025)

On the Diminishing Returns of Width for Continual Learning
by: Guha, Etash, et al.
Published: (2024)

When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs
by: Jeong, Soyeong, et al.
Published: (2025)

The PokeAgent Challenge: Competitive and Long-Context Learning at Scale
by: Karten, Seth, et al.
Published: (2026)

Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs
by: Song, Woomin, et al.
Published: (2024)

Benchmarking the Computational and Representational Efficiency of State Space Models against Transformers on Long-Context Dyadic Sessions
by: Koledoye, Abidemi, et al.
Published: (2026)

Deep Learning and Data Augmentation for Detecting Self-Admitted Technical Debt
by: Sutoyo, Edi, et al.
Published: (2024)

KV Admission: Learning What to Write for Efficient Long-Context Inference
by: Huang, Yen-Chieh, et al.
Published: (2025)

LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrations
by: Ruoss, Anian, et al.
Published: (2024)

LLoCO: Learning Long Contexts Offline
by: Tan, Sijun, et al.
Published: (2024)

MELODI: Exploring Memory Compression for Long Contexts
by: Chen, Yinpeng, et al.
Published: (2024)

PolicyLong: Towards On-Policy Context Extension
by: Jia, Junlong, et al.
Published: (2026)

Context and Diversity Matter: The Emergence of In-Context Learning in World Models
by: Wang, Fan, et al.
Published: (2025)

Reflective Context Learning: Studying the Optimization Primitives of Context Space
by: Vassilyev, Nikita, et al.
Published: (2026)

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning
by: Zhang, Kehao, et al.
Published: (2026)

100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?
by: Yang, Wang, et al.
Published: (2025)

ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RL
by: Lu, Xingyu, et al.
Published: (2026)

Overflow Prevention Enhances Long-Context Recurrent LLMs
by: Ben-Kish, Assaf, et al.
Published: (2025)

Breaking the Context Bottleneck on Long Time Series Forecasting
by: Ma, Chao, et al.
Published: (2024)

In-Context Reinforcement Learning through Bayesian Fusion of Context and Value Prior
by: Berkes, Anaïs, et al.
Published: (2026)

Automated Structured Radiology Report Generation with Rich Clinical Context
by: Kang, Seongjae, et al.
Published: (2025)

Learning Long-Context Diffusion Policies via Past-Token Prediction
by: Torne, Marcel, et al.
Published: (2025)

PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents
by: Gu, Zhuohan, et al.
Published: (2026)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
by: Li, Zeju, et al.
Published: (2026)

In-Context Learning for Pure Exploration
by: Russo, Alessio, et al.
Published: (2025)

Scalable In-Context Q-Learning
by: Liu, Jinmei, et al.
Published: (2025)

A Framework for Quantifying How Pre-Training and Context Benefit In-Context Learning
by: Song, Bingqing, et al.
Published: (2025)

Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks
by: Campbell, Ryan, et al.
Published: (2024)

Hydra: A Modular Architecture for Efficient Long-Context Reasoning
by: Chaudhary, Siddharth, et al.
Published: (2025)

Evaluating Long-Context Reasoning in LLM-Based WebAgents
by: Chung, Andy, et al.
Published: (2025)