Cover Image

Modular Techniques for Synthetic Long-Context Data Generation in Language Model Training and Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Subramanian, Seganrasan, Verma, Abhigya
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FABRIC: Framework for Agent-Based Realistic Intelligence Creation
by: Verma, Abhigya, et al.
Published: (2025)

GRAFT: GRaPH and Table Reasoning for Textual Alignment -- A Benchmark for Structured Instruction Following and Visual Reasoning
by: Verma, Abhigya, et al.
Published: (2025)

Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Models
by: Tian, Junfeng, et al.
Published: (2024)

LongSafety: Evaluating Long-Context Safety of Large Language Models
by: Lu, Yida, et al.
Published: (2025)

Agent-Centric Projection of Prompting Techniques and Implications for Synthetic Training Data for Large Language Models
by: Dhamani, Dhruv, et al.
Published: (2025)

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
by: Yen, Howard, et al.
Published: (2024)

PILOT: Steering Synthetic Data Generation with Psychological & Linguistic Output Targeting
by: Cisar, Caitlin, et al.
Published: (2025)

Grounding Synthetic Data Evaluations of Language Models in Unsupervised Document Corpora
by: Majurski, Michael, et al.
Published: (2025)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
by: Li, Jiaqi, et al.
Published: (2023)

CUB: Benchmarking Context Utilisation Techniques for Language Models
by: Hagström, Lovisa, et al.
Published: (2025)

LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
by: Zhao, Liang, et al.
Published: (2024)

How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models
by: Kim, Minsung, et al.
Published: (2025)

Synthetic Data Generation for Phrase Break Prediction with Large Language Model
by: Lee, Hoyeon, et al.
Published: (2025)

Training and Evaluating Language Models with Template-based Data Generation
by: Zhang, Yifan
Published: (2024)

Can Small Language Models Handle Context-Summarized Multi-Turn Customer-Service QA? A Synthetic Data-Driven Comparative Evaluation
by: Cooray, Lakshan, et al.
Published: (2026)

CEI: A Benchmark for Evaluating Pragmatic Reasoning in Language Models
by: Chun, Jon, et al.
Published: (2026)

Lost-in-the-Middle in Long-Text Generation: Synthetic Dataset, Evaluation Framework, and Mitigation
by: Zhang, Junhao, et al.
Published: (2025)

Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generation
by: Rousseau, Cécile, et al.
Published: (2025)

Scaling Laws of Synthetic Data for Language Models
by: Qin, Zeyu, et al.
Published: (2025)

Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data
by: Cook, John, et al.
Published: (2026)

Dynamic Context Evolution for Scalable Synthetic Data Generation
by: Lingo, Ryan, et al.
Published: (2026)

Retrieval meets Long Context Large Language Models
by: Xu, Peng, et al.
Published: (2023)

NExtLong: Toward Effective Long-Context Training without Long Documents
by: Gao, Chaochen, et al.
Published: (2025)

XL-Suite: Cross-Lingual Synthetic Training and Evaluation Data for Open-Ended Generation
by: Iyer, Vivek, et al.
Published: (2025)

Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning
by: Zhu, Wenhao, et al.
Published: (2025)

A Multi-Faceted Evaluation Framework for Assessing Synthetic Data Generated by Large Language Models
by: Yuan, Yefeng, et al.
Published: (2024)

EntropyLong: Effective Long-Context Training via Predictive Uncertainty
by: Jia, Junlong, et al.
Published: (2025)

Context Memorization for Efficient Long Context Generation
by: Okoshi, Yasuyuki, et al.
Published: (2026)

Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
by: Hu, Xiang, et al.
Published: (2025)

MeVe: A Modular System for Memory Verification and Effective Context Control in Language Models
by: Ottem, Andreas
Published: (2025)

ACC: Compiling Agent Trajectories for Long-Context Training
by: Su, Qisheng, et al.
Published: (2026)

Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
by: He, Linda, et al.
Published: (2025)

OPSDL: On-Policy Self-Distillation for Long-Context Language Models
by: Zhang, Xinsen, et al.
Published: (2026)

Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
by: Ma, Shengjie, et al.
Published: (2025)

Revisiting In-Context Learning with Long Context Language Models
by: Baek, Jinheon, et al.
Published: (2024)

Reasoning-Driven Synthetic Data Generation and Evaluation
by: Davidson, Tim R., et al.
Published: (2026)

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
by: Seddik, Mohamed El Amine, et al.
Published: (2024)

Self-Boosting Large Language Models with Synthetic Preference Data
by: Dong, Qingxiu, et al.
Published: (2024)

Configurable Safety Tuning of Language Models with Synthetic Preference Data
by: Gallego, Victor
Published: (2024)

Citekit: A Modular Toolkit for Large Language Model Citation Generation
by: Shen, Jiajun, et al.
Published: (2024)