Cover Image

Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yu, Yijiong, Huang, Yongfeng, Qi, Zhixiao, Wang, Wei, Liu, Weifeng, Chen, Ran, Pei, Ji
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Training With "Paraphrasing the Original Text" Teaches LLM to Better Retrieve in Long-context Tasks
by: Yu, Yijiong, et al.
Published: (2023)

An Effective Framework to Help Large Language Models Handle Numeric-involved Long-context Tasks
by: Yu, Yijiong
Published: (2024)

Patience Is The Key to Large Language Model Reasoning
by: Yu, Yijiong
Published: (2024)

Do LLMs Really Think Step-by-step In Implicit Reasoning?
by: Yu, Yijiong
Published: (2024)

Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence
by: Yu, Yijiong
Published: (2025)

OpenCSG Chinese Corpus: A Series of High-quality Chinese Datasets for LLM Training
by: Yu, Yijiong, et al.
Published: (2025)

SWAA: Sliding Window Attention Adaptation for Efficient and Quality Preserving Long Context Processing
by: Yu, Yijiong, et al.
Published: (2025)

Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models
by: Qiu, Yifu, et al.
Published: (2025)

Long Is More Important Than Difficult for Training Reasoning Models
by: Shen, Si, et al.
Published: (2025)

The Impact of Reasoning Step Length on Large Language Models
by: Jin, Mingyu, et al.
Published: (2024)

Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio
by: Yu, Yijiong, et al.
Published: (2026)

Speculative Pipeline Decoding: Higher-Accruacy and Zero-Bubble Speculation via Pipeline Parallelism
by: Yu, Yijiong, et al.
Published: (2026)

SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning
by: Huang, Yongfeng, et al.
Published: (2026)

Mitigate Position Bias in Large Language Models via Scaling a Single Dimension
by: Yu, Yijiong, et al.
Published: (2024)

Task-Aligned Tool Recommendation for Large Language Models
by: Gao, Hang, et al.
Published: (2024)

Large Language Models Can Self-Improve in Long-context Reasoning
by: Li, Siheng, et al.
Published: (2024)

STEPER: Step-wise Knowledge Distillation for Enhancing Reasoning Ability in Multi-Step Retrieval-Augmented Language Models
by: Lee, Kyumin, et al.
Published: (2025)

ALR$^2$: A Retrieve-then-Reason Framework for Long-context Question Answering
by: Li, Huayang, et al.
Published: (2024)

Fine-grained Stateful Knowledge Exploration: Effective and Efficient Graph Retrieval with Large Language Models
by: Tao, Dehao, et al.
Published: (2024)

Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Models
by: Wu, Junjie, et al.
Published: (2025)

Large Language Models are In-context Teachers for Knowledge Reasoning
by: Zhao, Jiachen, et al.
Published: (2023)

Hierarchical Document Refinement for Long-context Retrieval-augmented Generation
by: Jin, Jiajie, et al.
Published: (2025)

Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge
by: Feng, Yiyang, et al.
Published: (2026)

LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
by: Jiang, Ziyan, et al.
Published: (2024)

Large Language Model for Discrete Optimization Problems: Evaluation and Step-by-step Reasoning
by: Qian, Tianhao, et al.
Published: (2026)

Structured In-context Environment Scaling for Large Language Model Reasoning
by: Yu, Peng, et al.
Published: (2025)

Structure-Grounded Knowledge Retrieval via Code Dependencies for Multi-Step Data Reasoning
by: Huang, Xinyi
Published: (2026)

Improving Retrieval Augmented Language Model with Self-Reasoning
by: Xia, Yuan, et al.
Published: (2024)

RAISE: Enhancing Scientific Reasoning in LLMs via Step-by-Step Retrieval
by: Oh, Minhae, et al.
Published: (2025)

From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs
by: He, Jie, et al.
Published: (2025)

Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models
by: Yu, Bin, et al.
Published: (2025)

Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generation
by: Chen, Guo, et al.
Published: (2025)

R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning
by: Li, Yuan, et al.
Published: (2025)

Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models
by: Zhang, Qingjie, et al.
Published: (2025)

Reinforcing Compositional Retrieval: Retrieving Step-by-Step for Composing Informative Contexts
by: Long, Quanyu, et al.
Published: (2025)

LongReasonArena: A Long Reasoning Benchmark for Large Language Models
by: Ding, Jiayu, et al.
Published: (2025)

Evaluating Multilingual Long-Context Models for Retrieval and Reasoning
by: Agrawal, Ameeta, et al.
Published: (2024)

Multi-Step Semantic Reasoning in Generative Retrieval
by: Dong, Steven, et al.
Published: (2026)

Is Sarcasm Detection A Step-by-Step Reasoning Process in Large Language Models?
by: Yao, Ben, et al.
Published: (2024)

RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners
by: Hu, Chi, et al.
Published: (2024)