Cover Image

Long-context LLMs Struggle with Long In-context Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Tianle, Zhang, Ge, Do, Quy Duc, Yue, Xiang, Chen, Wenhu
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
by: Jiang, Ziyan, et al.
Published: (2024)

LongGenBench: Long-context Generation Benchmark
by: Liu, Xiang, et al.
Published: (2024)

A Decomposition Perspective to Long-context Reasoning for LLMs
by: Xiao, Yanling, et al.
Published: (2026)

Revisiting Long-context Modeling from Context Denoising Perspective
by: Tang, Zecheng, et al.
Published: (2025)

LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
by: Bai, Yushi, et al.
Published: (2024)

LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
by: Gao, Chaochen, et al.
Published: (2025)

AllMem: A Memory-centric Recipe for Efficient Long-context Modeling
by: Wang, Ziming, et al.
Published: (2026)

Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models
by: Qiu, Yifu, et al.
Published: (2025)

Large Language Models Can Self-Improve in Long-context Reasoning
by: Li, Siheng, et al.
Published: (2024)

AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies
by: Ye, Xiao, et al.
Published: (2024)

Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
by: Wang, Tianle, et al.
Published: (2026)

A Survey on In-context Learning
by: Dong, Qingxiu, et al.
Published: (2022)

Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks
by: Wang, Chonghua, et al.
Published: (2024)

Training With "Paraphrasing the Original Text" Teaches LLM to Better Retrieve in Long-context Tasks
by: Yu, Yijiong, et al.
Published: (2023)

Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs
by: Zhang, Haozhen, et al.
Published: (2024)

Quest: Query-centric Data Synthesis Approach for Long-context Scaling of Large Language Model
by: Gao, Chaochen, et al.
Published: (2024)

Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysis
by: Hu, Bo, et al.
Published: (2025)

VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation
by: Ni, Yuansheng, et al.
Published: (2025)

Are Long-LLMs A Necessity For Long-Context Tasks?
by: Qian, Hongjin, et al.
Published: (2024)

Prediction hubs are context-informed frequent tokens in LLMs
by: Nielsen, Beatrix M. G., et al.
Published: (2025)

Rethinking Invariance in In-context Learning
by: Fang, Lizhe, et al.
Published: (2025)

Implicit In-context Learning
by: Li, Zhuowei, et al.
Published: (2024)

LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
by: Jia, Junlong, et al.
Published: (2025)

In-context Language Learning for Endangered Languages in Speech Recognition
by: Li, Zhaolin, et al.
Published: (2025)

Multimodal In-context Learning for ASR of Low-resource Languages
by: Li, Zhaolin, et al.
Published: (2026)

Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering
by: Wang, Yubo, et al.
Published: (2023)

Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questions
by: Li, Xiang, et al.
Published: (2024)

Provable Benefits of Task-Specific Prompts for In-context Learning
by: Chang, Xiangyu, et al.
Published: (2025)

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking
by: Hegde, Pradyoth, et al.
Published: (2025)

No Need for Explanations: LLMs can implicitly learn from mistakes in-context
by: Alazraki, Lisa, et al.
Published: (2025)

Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning
by: Wang, Haozhe, et al.
Published: (2025)

A Study on the Calibration of In-context Learning
by: Zhang, Hanlin, et al.
Published: (2023)

Frontier LLMs Still Struggle with Simple Reasoning Tasks
by: Malek, Alan, et al.
Published: (2025)

Supervised Knowledge Makes Large Language Models Better In-context Learners
by: Yang, Linyi, et al.
Published: (2023)

A Cognitive Writing Perspective for Constrained Long-Form Text Generation
by: Wan, Kaiyang, et al.
Published: (2025)

On the Relation between Sensitivity and Accuracy in In-context Learning
by: Chen, Yanda, et al.
Published: (2022)

In-context Learning with Retrieved Demonstrations for Language Models: A Survey
by: Luo, Man, et al.
Published: (2024)

Vector-ICL: In-context Learning with Continuous Vector Representations
by: Zhuang, Yufan, et al.
Published: (2024)

StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
by: Yang, Jialin, et al.
Published: (2025)

Atomic Calibration of LLMs in Long-Form Generations
by: Zhang, Caiqi, et al.
Published: (2024)