Imagen de Portada

Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Ferreira, Fabio
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How
por: Arango, Sebastian Pineda, et al.
Publicado: (2023)

The Finetuner's Fallacy: When to Pretrain with Your Finetuning Data
por: Baek, Christina, et al.
Publicado: (2026)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
por: Bethune, Louis, et al.
Publicado: (2025)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
por: Feng, Jiahai, et al.
Publicado: (2024)

OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning
por: Yue, Sheng, et al.
Publicado: (2024)

LLM-Inspired Pretrain-Then-Finetune for Small-Data, Large-Scale Optimization
por: Zhang, Zishi, et al.
Publicado: (2026)

TMI! Finetuned Models Leak Private Information from their Pretraining Data
por: Abascal, John, et al.
Publicado: (2023)

Finetune-Informed Pretraining Boosts Downstream Performance
por: Faysal, Atik, et al.
Publicado: (2026)

Transfer Learning for Finetuning Large Language Models
por: Strangmann, Tobias, et al.
Publicado: (2024)

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
por: Xie, Wanyun, et al.
Publicado: (2025)

Towards Active Synthetic Data Generation for Finetuning Language Models
por: Kessler, Samuel, et al.
Publicado: (2025)

Boosting Statistic Learning with Synthetic Data from Pretrained Large Models
por: Jiang, Jialong, et al.
Publicado: (2025)

On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm
por: Zhou, Zhanpeng, et al.
Publicado: (2024)

CIMGEN: Controlled Image Manipulation by Finetuning Pretrained Generative Models on Limited Data
por: Gudavalli, Chandrakanth, et al.
Publicado: (2024)

Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe
por: You, Chong, et al.
Publicado: (2025)

A Pretraining-Finetuning Computational Framework for Material Homogenization
por: Wang, Yizheng, et al.
Publicado: (2024)

Data-Efficient Sleep Staging with Synthetic Time Series Pretraining
por: Grieger, Niklas, et al.
Publicado: (2024)

Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
por: Wagenmaker, Andrew, et al.
Publicado: (2025)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
por: Somayajula, Sai Ashish, et al.
Publicado: (2024)

TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
por: Hong, Matthew M., et al.
Publicado: (2026)

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
por: Longpre, Shayne, et al.
Publicado: (2025)

RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting
por: Shi, Zhan, et al.
Publicado: (2025)

BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
por: DatologyAI, et al.
Publicado: (2025)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
por: Zhang, Biao, et al.
Publicado: (2024)

Less Finetuning, Better Retrieval: Rethinking LLM Adaptation for Biomedical Retrievers via Synthetic Data and Model Merging
por: Khattab, Sameh, et al.
Publicado: (2026)

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
por: Xiong, Zheyang, et al.
Publicado: (2024)

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
por: Liu, Yuxing, et al.
Publicado: (2026)

From Basic to Extra Features: Hypergraph Transformer Pretrain-then-Finetuning for Balanced Clinical Predictions on EHR
por: Xu, Ran, et al.
Publicado: (2024)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
por: Vasudeva, Bhavya, et al.
Publicado: (2026)

On Pretraining Data Diversity for Self-Supervised Learning
por: Hammoud, Hasan Abed Al Kader, et al.
Publicado: (2024)

RL-Guided Data Selection for Language Model Finetuning
por: Jha, Animesh, et al.
Publicado: (2025)

CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data
por: Xie, Shifeng, et al.
Publicado: (2025)

Learning Dynamics of VLM Finetuning
por: Zhang, Jusheng, et al.
Publicado: (2025)

Representation Finetuning for Continual Learning
por: Luo, Haihua, et al.
Publicado: (2026)

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
por: Itzhak, Itay, et al.
Publicado: (2025)

Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning
por: Chen, Wuyang, et al.
Publicado: (2024)

Synthetic Data Generation and Automated Multidimensional Data Labeling for AI/ML in General and Circular Coordinates
por: Williams, Alice, et al.
Publicado: (2024)

Weakly Supervised Pretraining and Multi-Annotator Supervised Finetuning for Facial Wrinkle Detection
por: Moon, Ik Jun, et al.
Publicado: (2024)

Exact Unlearning of Finetuning Data via Model Merging at Scale
por: Kuo, Kevin, et al.
Publicado: (2025)

Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning
por: Ishibashi, Yoichi, et al.
Publicado: (2025)