Cover Image

Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Ferreira, Fabio
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How
by: Arango, Sebastian Pineda, et al.
Published: (2023)

The Finetuner's Fallacy: When to Pretrain with Your Finetuning Data
by: Baek, Christina, et al.
Published: (2026)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
by: Bethune, Louis, et al.
Published: (2025)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
by: Feng, Jiahai, et al.
Published: (2024)

OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning
by: Yue, Sheng, et al.
Published: (2024)

LLM-Inspired Pretrain-Then-Finetune for Small-Data, Large-Scale Optimization
by: Zhang, Zishi, et al.
Published: (2026)

TMI! Finetuned Models Leak Private Information from their Pretraining Data
by: Abascal, John, et al.
Published: (2023)

Finetune-Informed Pretraining Boosts Downstream Performance
by: Faysal, Atik, et al.
Published: (2026)

Transfer Learning for Finetuning Large Language Models
by: Strangmann, Tobias, et al.
Published: (2024)

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
by: Xie, Wanyun, et al.
Published: (2025)

Towards Active Synthetic Data Generation for Finetuning Language Models
by: Kessler, Samuel, et al.
Published: (2025)

Boosting Statistic Learning with Synthetic Data from Pretrained Large Models
by: Jiang, Jialong, et al.
Published: (2025)

On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm
by: Zhou, Zhanpeng, et al.
Published: (2024)

CIMGEN: Controlled Image Manipulation by Finetuning Pretrained Generative Models on Limited Data
by: Gudavalli, Chandrakanth, et al.
Published: (2024)

Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe
by: You, Chong, et al.
Published: (2025)

A Pretraining-Finetuning Computational Framework for Material Homogenization
by: Wang, Yizheng, et al.
Published: (2024)

Data-Efficient Sleep Staging with Synthetic Time Series Pretraining
by: Grieger, Niklas, et al.
Published: (2024)

Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
by: Wagenmaker, Andrew, et al.
Published: (2025)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
by: Somayajula, Sai Ashish, et al.
Published: (2024)

TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
by: Hong, Matthew M., et al.
Published: (2026)

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
by: Longpre, Shayne, et al.
Published: (2025)

RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting
by: Shi, Zhan, et al.
Published: (2025)

BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
by: DatologyAI, et al.
Published: (2025)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
by: Zhang, Biao, et al.
Published: (2024)

Less Finetuning, Better Retrieval: Rethinking LLM Adaptation for Biomedical Retrievers via Synthetic Data and Model Merging
by: Khattab, Sameh, et al.
Published: (2026)

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
by: Xiong, Zheyang, et al.
Published: (2024)

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
by: Liu, Yuxing, et al.
Published: (2026)

From Basic to Extra Features: Hypergraph Transformer Pretrain-then-Finetuning for Balanced Clinical Predictions on EHR
by: Xu, Ran, et al.
Published: (2024)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
by: Vasudeva, Bhavya, et al.
Published: (2026)

On Pretraining Data Diversity for Self-Supervised Learning
by: Hammoud, Hasan Abed Al Kader, et al.
Published: (2024)

RL-Guided Data Selection for Language Model Finetuning
by: Jha, Animesh, et al.
Published: (2025)

CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data
by: Xie, Shifeng, et al.
Published: (2025)

Learning Dynamics of VLM Finetuning
by: Zhang, Jusheng, et al.
Published: (2025)

Representation Finetuning for Continual Learning
by: Luo, Haihua, et al.
Published: (2026)

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
by: Itzhak, Itay, et al.
Published: (2025)

Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning
by: Chen, Wuyang, et al.
Published: (2024)

Synthetic Data Generation and Automated Multidimensional Data Labeling for AI/ML in General and Circular Coordinates
by: Williams, Alice, et al.
Published: (2024)

Weakly Supervised Pretraining and Multi-Annotator Supervised Finetuning for Facial Wrinkle Detection
by: Moon, Ik Jun, et al.
Published: (2024)

Exact Unlearning of Finetuning Data via Model Merging at Scale
by: Kuo, Kevin, et al.
Published: (2025)

Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning
by: Ishibashi, Yoichi, et al.
Published: (2025)