Cover Image

Balancing Synthetic Data and Replay for Enhancing Task-Specific Capabilities

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Spiegelhalter, Urs, Franke, Jörg K. H., Hutter, Frank
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning in Compact Spaces with Approximately Normalized Transformer
by: Franke, Jörg K. H., et al.
Published: (2025)

Increasing LLM Coding Capabilities through Diverse Synthetic Coding Tasks
by: Abed, Amal, et al.
Published: (2025)

Transfer Learning for Finetuning Large Language Models
by: Strangmann, Tobias, et al.
Published: (2024)

Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues
by: Grazzi, Riccardo, et al.
Published: (2024)

Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs
by: Chan, Yung-Chieh, et al.
Published: (2024)

Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Data
by: Gill, Waris, et al.
Published: (2025)

RePO: Replay-Enhanced Policy Optimization
by: Li, Siheng, et al.
Published: (2025)

Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models
by: Jadon, Aryan, et al.
Published: (2025)

Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data
by: Patwa, Parth, et al.
Published: (2024)

CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
by: Ziegler, Ingo, et al.
Published: (2024)

MALTO at SemEval-2024 Task 6: Leveraging Synthetic Data for LLM Hallucination Detection
by: Borra, Federico, et al.
Published: (2024)

Synthetic Data RL: Task Definition Is All You Need
by: Guo, Yiduo, et al.
Published: (2025)

Biased or Flawed? Mitigating Stereotypes in Generative Language Models by Addressing Task-Specific Flaws
by: Jha, Akshita, et al.
Published: (2024)

Sample-efficient LLM Optimization with Reset Replay
by: Liu, Zichuan, et al.
Published: (2025)

Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge
by: Chen, Yan-Lun, et al.
Published: (2025)

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
by: Xiong, Zheyang, et al.
Published: (2024)

Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities
by: Zhang, Junyan, et al.
Published: (2025)

Replaying pre-training data improves fine-tuning
by: Kotha, Suhas, et al.
Published: (2026)

Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Data
by: Zhou, Jiaming, et al.
Published: (2024)

InCo-DPO: Balancing Distribution Shift and Data Quality for Enhanced Preference Optimization
by: Wang, Yunan, et al.
Published: (2025)

Tuning LLM Judge Design Decisions for 1/1000 of the Cost
by: Salinas, David, et al.
Published: (2025)

Improving Deep Learning Optimization through Constrained Parameter Regularization
by: Franke, Jörg K. H., et al.
Published: (2023)

Improving RL Exploration for LLM Reasoning through Retrospective Replay
by: Dou, Shihan, et al.
Published: (2025)

AAPO: Enhancing the Reasoning Capabilities of LLMs with Advantage Margin
by: Xiong, Jian, et al.
Published: (2025)

Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons
by: Leng, Yongqi, et al.
Published: (2024)

Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities
by: Sun, Chung-En, et al.
Published: (2024)

Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models
by: Eiras, Francisco, et al.
Published: (2024)

Efficacy of Synthetic Data as a Benchmark
by: Maheshwari, Gaurav, et al.
Published: (2024)

Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning
by: Ma, Weiyu, et al.
Published: (2026)

X-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Tests
by: Wu, Jie, et al.
Published: (2026)

ContinuousBench: Can Differentially Private Synthetic Text Improve Capabilities?
by: Liu, Peihan, et al.
Published: (2026)

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes
by: Bhasin, Harmon, et al.
Published: (2024)

Rethinking Performance Measures of RNA Secondary Structure Problems
by: Runge, Frederic, et al.
Published: (2023)

Prioritized Replay for RL Post-training
by: Fatemi, Mehdi
Published: (2026)

Capable but Unreliable: Canonical Path Deviation as a Causal Mechanism of Agent Failure in Long-Horizon Tasks
by: Lee, Wilson Y.
Published: (2026)

Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models
by: Rannen-Triki, Amal, et al.
Published: (2024)

ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning
by: Wu, Yang, et al.
Published: (2024)

Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning
by: Sarkar, Soumajyoti, et al.
Published: (2024)

Selective Rotary Position Embedding
by: Movahedi, Sajad, et al.
Published: (2025)

EduRABSA: An Education Review Dataset for Aspect-based Sentiment Analysis Tasks
by: Hua, Yan Cathy, et al.
Published: (2025)