Image de couverture de livre

The Finetuner's Fallacy: When to Pretrain with Your Finetuning Data

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Baek, Christina, Monti, Ricardo Pio, Schwab, David, Abbas, Amro, Adiga, Rishabh, Blakeney, Cody, Böther, Maximilian, Burstein, Paul, Carranza, Aldo Gael, Deng, Alvin, Doshi, Parth, Dorna, Vineeth, Fang, Alex, Jiang, Tony, Joshi, Siddharth, Larsen, Brett W., Lee, Jason Chan, Mentzer, Katherine L., Merrick, Luke, Mongstad, Haakon, Pan, Fan, Suri, Anshuman, Teh, Darren, Telanoff, Jason, Urbanek, Jack, Wang, Zhengping, Wills, Josh, Yin, Haoli, Raghunathan, Aditi, Kolter, J. Zico, Gaza, Bogdan, Morcos, Ari, Leavitt, Matthew, Maini, Pratyush
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

ÜberWeb: Insights from Multilingual Curation for a 20-Trillion-Token Dataset
par: DatologyAI, et autres
Publié: (2026)

DatBench: Discriminative, Faithful, and Efficient VLM Evaluations
par: DatologyAI, et autres
Publié: (2026)

Luxical: High-Speed Lexical-Dense Text Embeddings
par: DatologyAI, et autres
Publié: (2025)

20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
par: DatologyAI, et autres
Publié: (2026)

BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
par: DatologyAI, et autres
Publié: (2025)

Finetuning CLIP to Reason about Pairwise Differences
par: Sam, Dylan, et autres
Publié: (2024)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
par: Goyal, Sachin, et autres
Publié: (2024)

OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics
par: Dorna, Vineeth, et autres
Publié: (2025)

Understanding Hallucinations in Diffusion Models through Mode Interpolation
par: Aithal, Sumukh K, et autres
Publié: (2024)

When Should We Introduce Safety Interventions During Pretraining?
par: Sam, Dylan, et autres
Publié: (2026)

T-MARS: Improving Visual Representations by Circumventing Text Feature Learning
par: Maini, Pratyush, et autres
Publié: (2023)

Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic
par: Goyal, Sachin, et autres
Publié: (2024)

TOFU: A Task of Fictitious Unlearning for LLMs
par: Maini, Pratyush, et autres
Publié: (2024)

Rethinking LLM Memorization through the Lens of Adversarial Compression
par: Schwarzschild, Avi, et autres
Publié: (2024)

Mimetic Initialization of MLPs
par: Trockman, Asher, et autres
Publié: (2026)

AcceleratedLiNGAM: Learning Causal DAGs at the speed of GPUs
par: Akinwande, Victor, et autres
Publié: (2024)

FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers
par: Williams, Joshua Nathaniel, et autres
Publié: (2024)

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
par: Huang, Benhao, et autres
Publié: (2026)

Why is SAM Robust to Label Noise?
par: Baek, Christina, et autres
Publié: (2024)

Logits-Based Finetuning
par: Li, Jingyao, et autres
Publié: (2025)

Order Independence With Finetuning
par: Brown, Katrina, et autres
Publié: (2025)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
par: Zhang, Biao, et autres
Publié: (2024)

Understanding Optimization in Deep Learning with Central Flows
par: Cohen, Jeremy M., et autres
Publié: (2024)

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
par: Gupta, Prakhar, et autres
Publié: (2026)

Predicting the Performance of Black-box LLMs through Follow-up Queries
par: Sam, Dylan, et autres
Publié: (2025)

One-Step Diffusion Distillation via Deep Equilibrium Models
par: Geng, Zhengyang, et autres
Publié: (2023)

Diffusing Differentiable Representations
par: Savani, Yash, et autres
Publié: (2024)

UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback
par: Wu, Jason, et autres
Publié: (2024)

On Finetuning Tabular Foundation Models
par: Rubachev, Ivan, et autres
Publié: (2025)

Learning Dynamics of LLM Finetuning
par: Ren, Yi, et autres
Publié: (2024)

Orthogonal Finetuning Made Scalable
par: Qiu, Zeju, et autres
Publié: (2025)

Representation Finetuning for Continual Learning
par: Luo, Haihua, et autres
Publié: (2026)

Improving Sparse Memory Finetuning
par: Goyal, Satyam, et autres
Publié: (2026)

Iterative Finetuning is Mostly Idempotent
par: Roe, Zephaniah, et autres
Publié: (2026)

Exploring Representation Invariance in Finetuning
par: Zu, Wenqiang, et autres
Publié: (2025)

Predicting Emergent Capabilities by Finetuning
par: Snell, Charlie, et autres
Publié: (2024)

The Hallucination Tax of Reinforcement Finetuning
par: Song, Linxin, et autres
Publié: (2025)

Whisper Finetuning on Nepali Language
par: Rijal, Sanjay, et autres
Publié: (2024)

Learning Dynamics of VLM Finetuning
par: Zhang, Jusheng, et autres
Publié: (2025)

Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs
par: Tan, Bowen, et autres
Publié: (2025)