Image de couverture de livre

Overtrained Language Models Are Harder to Fine-Tune

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Springer, Jacob Mitchell, Goyal, Sachin, Wen, Kaiyue, Kumar, Tanishq, Yue, Xiang, Malladi, Sadhika, Neubig, Graham, Raghunathan, Aditi
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Repetition Improves Language Model Embeddings
par: Springer, Jacob Mitchell, et autres
Publié: (2024)

Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
par: Watts, Ishaan, et autres
Publié: (2026)

Understanding Catastrophic Forgetting in Language Models via Implicit Inference
par: Kotha, Suhas, et autres
Publié: (2023)

Mode-Conditioning Unlocks Superior Test-Time Scaling
par: Wu, Chen Henry, et autres
Publié: (2025)

LESS: Selecting Influential Data for Targeted Instruction Tuning
par: Xia, Mengzhou, et autres
Publié: (2024)

Fine-Tuning Language Models with Just Forward Passes
par: Malladi, Sadhika, et autres
Publié: (2023)

Understanding the Influence of Synthetic Data for Text Embedders
par: Springer, Jacob Mitchell, et autres
Publié: (2025)

Mitigating Bias in RAG: Controlling the Embedder
par: Kim, Taeyoun, et autres
Publié: (2025)

Weight Ensembling Improves Reasoning in Language Models
par: Dang, Xingyu, et autres
Publié: (2025)

On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
par: Zhang, Charlie, et autres
Publié: (2025)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
par: Goyal, Sachin, et autres
Publié: (2024)

Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions
par: Liu, Emmy, et autres
Publié: (2025)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
par: Razin, Noam, et autres
Publié: (2024)

MUSE: Machine Unlearning Six-Way Evaluation for Language Models
par: Shi, Weijia, et autres
Publié: (2024)

Language Modeling with Editable External Knowledge
par: Li, Belinda Z., et autres
Publié: (2024)

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models
par: Tjuatja, Lindia, et autres
Publié: (2025)

Early Data Exposure Improves Robustness to Subsequent Fine-Tuning
par: Feng, Lawrence, et autres
Publié: (2026)

Metadata Conditioning Accelerates Language Model Pre-training
par: Gao, Tianyu, et autres
Publié: (2025)

Self-Trained Verification for Training- and Test-Time Self-Improvement
par: Wu, Chen Henry, et autres
Publié: (2026)

Effective Strategies for Asynchronous Software Engineering Agents
par: Geng, Jiayi, et autres
Publié: (2026)

Trainable Transformer in Transformer
par: Panigrahi, Abhishek, et autres
Publié: (2023)

Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate
par: Chern, Steffi, et autres
Publié: (2024)

Preference Learning Algorithms Do Not Learn Preference Rankings
par: Chen, Angelica, et autres
Publié: (2024)

An Incomplete Loop: Instruction Inference, Instruction Following, and In-context Learning in Language Models
par: Liu, Emmy, et autres
Publié: (2024)

Natural Language Fine-Tuning
par: Liu, Jia, et autres
Publié: (2024)

What Are Tools Anyway? A Survey from the Language Model Perspective
par: Wang, Zhiruo, et autres
Publié: (2024)

ZINA: Multimodal Fine-grained Hallucination Detection and Editing
par: Wada, Yuiga, et autres
Publié: (2025)

Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning
par: Springer, Jacob Mitchell, et autres
Publié: (2024)

Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models
par: Kumar, Sachin
Publié: (2026)

Reusing Overtrained Language Models Saturates Scaling
par: Liew, Seng Pei, et autres
Publié: (2025)

G-Loss: Graph-Guided Fine-Tuning of Language Models
par: Sharma, Aditya, et autres
Publié: (2026)

Boosting Large Language Models with Mask Fine-Tuning
par: Zhang, Mingyuan, et autres
Publié: (2025)

Provable unlearning in topic modeling and downstream tasks
par: Wei, Stanley, et autres
Publié: (2024)

Flow Map Language Models: One-step Language Modeling via Continuous Denoising
par: Lee, Chanhyuk, et autres
Publié: (2026)

LLMs for Explainable Business Decision-Making: A Reinforcement Learning Fine-Tuning Approach
par: Cheng, Xiang, et autres
Publié: (2025)

Memorization in Fine-Tuned Large Language Models
par: Savine, Danil
Publié: (2025)

In Good GRACEs: Principled Teacher Selection for Knowledge Distillation
par: Panigrahi, Abhishek, et autres
Publié: (2025)

T-MARS: Improving Visual Representations by Circumventing Text Feature Learning
par: Maini, Pratyush, et autres
Publié: (2023)

The Coverage Principle: How Pre-Training Enables Post-Training
par: Chen, Fan, et autres
Publié: (2025)

Think before you speak: Training Language Models With Pause Tokens
par: Goyal, Sachin, et autres
Publié: (2023)