Cover Image

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xiong, Zheyang, Papageorgiou, Vasilis, Lee, Kangwook, Papailiopoulos, Dimitris
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
by: Xiong, Zheyang, et al.
Published: (2024)

Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
by: Lee, Nayoung, et al.
Published: (2025)

Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Find
by: Bianchi, Owen, et al.
Published: (2025)

From Haystack to Needle: Label Space Reduction for Zero-shot Classification
by: Vandemoortele, Nathan, et al.
Published: (2025)

Needle in the Haystack for Memory Based Large Language Models
by: Nelson, Elliot, et al.
Published: (2024)

In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss
by: Kuratov, Yuri, et al.
Published: (2024)

DENIAHL: In-Context Features Influence LLM Needle-In-A-Haystack Abilities
by: Dai, Hui, et al.
Published: (2024)

Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models
by: Wang, Hengyi, et al.
Published: (2024)

Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding
by: Yang, Seongjun, et al.
Published: (2023)

The Expressive Power of Low-Rank Adaptation
by: Zeng, Yuchen, et al.
Published: (2023)

Semantic Needles in Document Haystacks: Sensitivity Testing of LLM-as-a-Judge Similarity Scoring
by: Aksoy, Sinan G., et al.
Published: (2026)

Artificial Conversations, Real Results: Fostering Language Detection with Synthetic Data
by: Mohammadi, Fatemeh, et al.
Published: (2025)

Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
by: Zhang, Hanlin, et al.
Published: (2026)

MEMENTO: Teaching LLMs to Manage Their Own Context
by: Kontonis, Vasilis, et al.
Published: (2026)

Jailbreaking in the Haystack
by: Shah, Rishi Rajesh, et al.
Published: (2025)

Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learning
by: Jin, Jikai, et al.
Published: (2025)

Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks
by: Park, Jongho, et al.
Published: (2024)

Reasoning on Multiple Needles In A Haystack
by: Wang, Yidong
Published: (2025)

Learning from Synthetic Data Improves Multi-hop Reasoning
by: Kabra, Anmol, et al.
Published: (2026)

Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
by: Pham, Bao, et al.
Published: (2026)

Programming by Backprop: An Instruction is Worth 100 Examples When Finetuning LLMs
by: Cook, Jonathan, et al.
Published: (2025)

Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets
by: Indurthi, Sathish Reddy, et al.
Published: (2024)

Contrastive Learning to Improve Retrieval for Real-world Fact Checking
by: Sriram, Aniruddh, et al.
Published: (2024)

Not All Synthetic Data Is Yours to Learn From
by: Alemohammad, Sina, et al.
Published: (2026)

Transfer Learning of Tabular Data by Finetuning Large Language Models
by: Rabbani, Shourav B., et al.
Published: (2025)

Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs
by: Zhang, Yifei, et al.
Published: (2024)

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
by: Itzhak, Itay, et al.
Published: (2025)

Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack
by: Xu, Xiaoyue, et al.
Published: (2024)

Beyond Needle(s) in the Embodied Haystack: Environment, Architecture, and Training Considerations for Long Context Reasoning
by: Kim, Bosung, et al.
Published: (2025)

Order Independence With Finetuning
by: Brown, Katrina, et al.
Published: (2025)

Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky
by: Hathidara, Ashutosh, et al.
Published: (2025)

MisSynth: Improving MISSCI Logical Fallacies Classification with Synthetic Data
by: Poliakov, Mykhailo, et al.
Published: (2025)

Seeing the Needle in the Haystack: Towards Weakly-Supervised Log Instance Anomaly Localization via Counterfactual Perturbation
by: Wong, Yutszyuk, et al.
Published: (2026)

Quantifying the Capabilities of LLMs across Scale and Precision
by: Badshah, Sher, et al.
Published: (2024)

Expanding Foundational Language Capabilities in Open-Source LLMs through a Korean Case Study
by: Lim, Junghwan, et al.
Published: (2025)

On Calibration of Large Language Models: From Response To Capability
by: Yang, Sin-Han, et al.
Published: (2026)

Enhancing Clinical Documentation with Synthetic Data: Leveraging Generative Models for Improved Accuracy
by: Biswas, Anjanava, et al.
Published: (2024)

Learning Dynamics of LLM Finetuning
by: Ren, Yi, et al.
Published: (2024)

A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs
by: Lin, Yihan, et al.
Published: (2025)

Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models
by: Jadon, Aryan, et al.
Published: (2025)