Copertina

Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	He, Linda, Wang, Jue, Weber, Maurice, Zhu, Shang, Athiwaratkun, Ben, Zhang, Ce
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale
di: Liu, Jinghui, et al.
Pubblicazione: (2026)

Token Alignment via Character Matching for Subword Completion
di: Athiwaratkun, Ben, et al.
Pubblicazione: (2024)

Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning
di: Zhu, Wenhao, et al.
Pubblicazione: (2025)

HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation
di: Jiao, YiHan, et al.
Pubblicazione: (2025)

GemmAr: Enhancing LLMs Through Arabic Instruction-Tuning
di: Chouikhi, Hasna, et al.
Pubblicazione: (2024)

GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation
di: Chen, Zihong, et al.
Pubblicazione: (2025)

Data Diversification Methods In Alignment Enhance Math Performance In LLMs
di: Dokmeci, Berkan, et al.
Pubblicazione: (2025)

Span-level Emotion-Cause-Category Triplet Extraction with Instruction Tuning LLMs and Data Augmentation
di: Li, Xiangju, et al.
Pubblicazione: (2025)

Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs
di: Tavakoli, Mohammad, et al.
Pubblicazione: (2025)

A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
di: Corbeil, Jean-Philippe, et al.
Pubblicazione: (2025)

Jailbreak Instruction-Tuned LLMs via end-of-sentence MLP Re-weighting
di: Luo, Yifan, et al.
Pubblicazione: (2024)

Next-Token Prediction Task Assumes Optimal Data Ordering for LLM Training in Proof Generation
di: An, Chenyang, et al.
Pubblicazione: (2024)

OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset
di: Toshniwal, Shubham, et al.
Pubblicazione: (2024)

One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness
di: Potraghloo, Erfan Baghaei, et al.
Pubblicazione: (2026)

Refract ICL: Rethinking Example Selection in the Era of Million-Token Models
di: Akula, Arjun R., et al.
Pubblicazione: (2025)

Toward Graph-Tokenizing Large Language Models with Reconstructive Graph Instruction Tuning
di: Zhang, Zhongjian, et al.
Pubblicazione: (2026)

Unveiling the Impact of Coding Data Instruction Fine-Tuning on Large Language Models Reasoning
di: Zhang, Xinlu, et al.
Pubblicazione: (2024)

Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact
di: Lee, Hyunji, et al.
Pubblicazione: (2025)

ClimateChat: Designing Data and Methods for Instruction Tuning LLMs to Answer Climate Change Queries
di: Chen, Zhou, et al.
Pubblicazione: (2025)

Staircase Streaming for Low-Latency Multi-Agent Inference
di: Wang, Junlin, et al.
Pubblicazione: (2025)

Know When To Fold 'Em: Token-Efficient LLM Synthetic Data Generation via Multi-Stage In-Flight Rejection
di: Chowdhury, Anjir Ahmed, et al.
Pubblicazione: (2026)

Step-On-Feet Tuning: Scaling Self-Alignment of LLMs via Bootstrapping
di: Wang, Haoyu, et al.
Pubblicazione: (2024)

How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs?
di: Doostmohammadi, Ehsan, et al.
Pubblicazione: (2024)

When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
di: Xu, Zhen, et al.
Pubblicazione: (2025)

Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis
di: Yuan, Lin, et al.
Pubblicazione: (2025)

DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
di: Zhou, Ying, et al.
Pubblicazione: (2024)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
di: Pang, Jinlong, et al.
Pubblicazione: (2025)

Dynamic Context Evolution for Scalable Synthetic Data Generation
di: Lingo, Ryan, et al.
Pubblicazione: (2026)

MathScale: Scaling Instruction Tuning for Mathematical Reasoning
di: Tang, Zhengyang, et al.
Pubblicazione: (2024)

T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning
di: Fu, Yanjun, et al.
Pubblicazione: (2025)

Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning
di: Zan, Changtong, et al.
Pubblicazione: (2024)

Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
di: Li, Ming, et al.
Pubblicazione: (2024)

TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection
di: He, Xixiang, et al.
Pubblicazione: (2025)

CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning
di: Lu, Zhiyuan, et al.
Pubblicazione: (2026)

Controllable Text Generation in the Instruction-Tuning Era
di: Ashok, Dhananjay, et al.
Pubblicazione: (2024)

Configurable Preference Tuning with Rubric-Guided Synthetic Data
di: Gallego, Víctor
Pubblicazione: (2025)

SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
di: Liu, Liangxin, et al.
Pubblicazione: (2024)

EMCEE: Improving Multilingual Capability of LLMs via Bridging Knowledge and Reasoning with Extracted Synthetic Multilingual Context
di: Koo, Hamin, et al.
Pubblicazione: (2025)

Federated Data-Efficient Instruction Tuning for Large Language Models
di: Qin, Zhen, et al.
Pubblicazione: (2024)

Not All Documents Are What You Need for Extracting Instruction Tuning Data
di: Zhang, Chi, et al.
Pubblicazione: (2025)