Cover Image

In-Context Learning with Long-Context Models: An In-Depth Exploration

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bertsch, Amanda, Ivgi, Maor, Xiao, Emily, Alon, Uri, Berant, Jonathan, Gormley, Matthew R., Neubig, Graham
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities
by: Bertsch, Amanda, et al.
Published: (2025)

From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty
by: Ivgi, Maor, et al.
Published: (2024)

Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention
by: Xiao, Emily, et al.
Published: (2025)

A Taxonomy for Data Contamination in Large Language Models
by: Palavalli, Medha, et al.
Published: (2024)

Prompt-MII: Meta-Learning Instruction Induction for LLMs
by: Xiao, Emily, et al.
Published: (2025)

Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval
by: Rubin, Ohad, et al.
Published: (2023)

Making Retrieval-Augmented Language Models Robust to Irrelevant Context
by: Yoran, Ori, et al.
Published: (2023)

Go-Browse: Training Web Agents with Structured Exploration
by: Gandhi, Apurva, et al.
Published: (2025)

From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models
by: Welleck, Sean, et al.
Published: (2024)

Better Instruction-Following Through Minimum Bayes Risk
by: Wu, Ian, et al.
Published: (2024)

In-Context Principle Learning from Mistakes
by: Zhang, Tianjun, et al.
Published: (2024)

Multi-Dimensional Evaluation of Text Summarization with In-Context Learning
by: Jain, Sameer, et al.
Published: (2023)

Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors
by: Amos, Ido, et al.
Published: (2023)

DoubleDipper: Improving Long-Context LLMs via Context Recycling
by: Cattan, Arie, et al.
Published: (2024)

Accumulating Context Changes the Beliefs of Language Models
by: Geng, Jiayi, et al.
Published: (2025)

Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP
by: Goldman, Omer, et al.
Published: (2024)

What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration
by: Qin, Libo, et al.
Published: (2024)

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models
by: Tjuatja, Lindia, et al.
Published: (2025)

In-Context Learning on a Budget: A Case Study in Token Classification
by: Berger, Uri, et al.
Published: (2024)

An Incomplete Loop: Instruction Inference, Instruction Following, and In-context Learning in Language Models
by: Liu, Emmy, et al.
Published: (2024)

LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
by: Ji, Mengmeng, et al.
Published: (2026)

Learning Mutually Informed Representations for Characters and Subwords
by: Wang, Yilin, et al.
Published: (2023)

Answering Questions by Meta-Reasoning over Multiple Chains of Thought
by: Yoran, Ori, et al.
Published: (2023)

On Many-Shot In-Context Learning for Long-Context Evaluation
by: Zou, Kaijian, et al.
Published: (2024)

Long-Context Language Modeling with Parallel Context Encoding
by: Yen, Howard, et al.
Published: (2024)

LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
by: Chen, Guanzheng, et al.
Published: (2026)

FicSim: A Dataset for Multi-Faceted Semantic Similarity in Long-Form Fiction
by: Johnson, Natasha, et al.
Published: (2025)

Revisiting In-Context Learning with Long Context Language Models
by: Baek, Jinheon, et al.
Published: (2024)

Large Language Models for Psycholinguistic Plausibility Pretesting
by: Amouyal, Samuel Joseph, et al.
Published: (2024)

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
by: Shyam, Vasudev, et al.
Published: (2024)

On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
by: Zhang, Charlie, et al.
Published: (2025)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
by: You, Zeng, et al.
Published: (2025)

Reducing Distraction in Long-Context Language Models by Focused Learning
by: Wu, Zijun, et al.
Published: (2024)

Effective Strategies for Asynchronous Software Engineering Agents
by: Geng, Jiayi, et al.
Published: (2026)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
by: Li, Jiaqi, et al.
Published: (2023)

Comparing Human and Language Models Sentence Processing Difficulties on Complex Structures
by: Amouyal, Samuel Joseph, et al.
Published: (2025)

RULER: What's the Real Context Size of Your Long-Context Language Models?
by: Hsieh, Cheng-Ping, et al.
Published: (2024)

Core Context Aware Transformers for Long Context Language Modeling
by: Chen, Yaofo, et al.
Published: (2024)

Weakly Supervised Text-to-SQL Parsing through Question Decomposition
by: Wolfson, Tomer, et al.
Published: (2021)

In-Context Learning for Long-Context Sentiment Analysis on Infrastructure Project Opinions
by: Shamshiri, Alireza, et al.
Published: (2024)