Image de couverture de livre

Power-Law Decay Loss for Large Language Model Finetuning: A Theory Perspective

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Shao, Jintian
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Transfer Learning for Finetuning Large Language Models
par: Strangmann, Tobias, et autres
Publié: (2024)

Large Language Models to Diffusion Finetuning
par: Cetin, Edoardo, et autres
Publié: (2025)

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector
par: Zhang, Andi, et autres
Publié: (2024)

Towards Analyzing and Understanding the Limitations of VAPO: A Theoretical Perspective
par: Shao, Jintian, et autres
Publié: (2025)

MuonAll: Muon Variant for Efficient Finetuning of Large Language Models
par: Page, Saurabh, et autres
Publié: (2025)

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
par: Liao, Baohao, et autres
Publié: (2024)

CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models
par: Gong, Zi, et autres
Publié: (2024)

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective
par: Wen, Kaiyue, et autres
Publié: (2024)

Ensembling Finetuned Language Models for Text Classification
par: Arango, Sebastian Pineda, et autres
Publié: (2024)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
par: Bethune, Louis, et autres
Publié: (2025)

Performance Law of Large Language Models
par: Wu, Chuhan, et autres
Publié: (2024)

Transfer Learning of Tabular Data by Finetuning Large Language Models
par: Rabbani, Shourav B., et autres
Publié: (2025)

Finetuning Language Models to Emit Linguistic Expressions of Uncertainty
par: Chaudhry, Arslan, et autres
Publié: (2024)

Towards Active Synthetic Data Generation for Finetuning Language Models
par: Kessler, Samuel, et autres
Publié: (2025)

Delta Activations: A Representation for Finetuned Large Language Models
par: Xu, Zhiqiu, et autres
Publié: (2025)

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
par: Xie, Wanyun, et autres
Publié: (2025)

Cut Your Losses in Large-Vocabulary Language Models
par: Wijmans, Erik, et autres
Publié: (2024)

Scaling Laws for Discriminative Classification in Large Language Models
par: Wyatte, Dean, et autres
Publié: (2024)

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning
par: Liu, Z, et autres
Publié: (2024)

An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
par: Bhatt, Gantavya, et autres
Publié: (2024)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
par: Zhang, Biao, et autres
Publié: (2024)

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
par: Longpre, Shayne, et autres
Publié: (2025)

Reranking Laws for Language Generation: A Communication-Theoretic Perspective
par: Farinhas, António, et autres
Publié: (2024)

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training
par: Bergsma, Shane, et autres
Publié: (2025)

PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
par: Hayou, Soufiane, et autres
Publié: (2025)

Vanishing Gradients in Reinforcement Finetuning of Language Models
par: Razin, Noam, et autres
Publié: (2023)

ReFT: Representation Finetuning for Language Models
par: Wu, Zhengxuan, et autres
Publié: (2024)

Scaling Laws for Post Training Quantized Large Language Models
par: Xu, Zifei, et autres
Publié: (2024)

Scaling Laws for Downstream Task Performance of Large Language Models
par: Isik, Berivan, et autres
Publié: (2024)

Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model
par: Liang, Jing, et autres
Publié: (2025)

CARE-RFT: Confidence-Anchored Reinforcement Finetuning for Reliable Reasoning in Large Language Models
par: Li, Shuozhe, et autres
Publié: (2026)

Exploring Scaling Laws for Local SGD in Large Language Model Training
par: He, Qiaozhi, et autres
Publié: (2024)

Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare
par: Gondara, Lovedeep, et autres
Publié: (2025)

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules
par: Luo, Kairong, et autres
Publié: (2025)

Understanding Emergent Abilities of Language Models from the Loss Perspective
par: Du, Zhengxiao, et autres
Publié: (2024)

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
par: Huang, Yangyi, et autres
Publié: (2026)

LawLLM: Law Large Language Model for the US Legal System
par: Shu, Dong, et autres
Publié: (2024)

Finetuning Large Language Models for Automated Depression Screening in Nigerian Pidgin English: GENSCORE Pilot Study
par: Olufadewa, Isaac Iyinoluwa, et autres
Publié: (2025)

CoT is Not True Reasoning, It Is Just a Tight Constraint to Imitate: A Theory Perspective
par: Shao, Jintian, et autres
Publié: (2025)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
par: Kang, Katie, et autres
Publié: (2024)