Imagen de Portada

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Biao, Liu, Zhongtao, Cherry, Colin, Firat, Orhan
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Scaling Sign Language Translation
por: Zhang, Biao, et al.
Publicado: (2024)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
por: Bethune, Louis, et al.
Publicado: (2025)

Learning Dynamics of LLM Finetuning
por: Ren, Yi, et al.
Publicado: (2024)

Towards Active Synthetic Data Generation for Finetuning Language Models
por: Kessler, Samuel, et al.
Publicado: (2025)

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
por: Gupta, Prakhar, et al.
Publicado: (2026)

Predicting Emergent Capabilities by Finetuning
por: Snell, Charlie, et al.
Publicado: (2024)

Improving Sparse Memory Finetuning
por: Goyal, Satyam, et al.
Publicado: (2026)

Ensembling Finetuned Language Models for Text Classification
por: Arango, Sebastian Pineda, et al.
Publicado: (2024)

Transfer Learning for Finetuning Large Language Models
por: Strangmann, Tobias, et al.
Publicado: (2024)

ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections
por: Bini, Massimo, et al.
Publicado: (2024)

Less Finetuning, Better Retrieval: Rethinking LLM Adaptation for Biomedical Retrievers via Synthetic Data and Model Merging
por: Khattab, Sameh, et al.
Publicado: (2026)

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
por: Xie, Wanyun, et al.
Publicado: (2025)

Order Independence With Finetuning
por: Brown, Katrina, et al.
Publicado: (2025)

Finetuning Language Models to Emit Linguistic Expressions of Uncertainty
por: Chaudhry, Arslan, et al.
Publicado: (2024)

Sparse Upcycling: Inference Inefficient Finetuning
por: Doubov, Sasha, et al.
Publicado: (2024)

Understanding Finetuning for Factual Knowledge Extraction
por: Ghosal, Gaurav, et al.
Publicado: (2024)

Large Language Models to Diffusion Finetuning
por: Cetin, Edoardo, et al.
Publicado: (2025)

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
por: Oliaro, Gabriele, et al.
Publicado: (2024)

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning
por: Liu, Z, et al.
Publicado: (2024)

Transfer Learning of Tabular Data by Finetuning Large Language Models
por: Rabbani, Shourav B., et al.
Publicado: (2025)

Programming by Backprop: An Instruction is Worth 100 Examples When Finetuning LLMs
por: Cook, Jonathan, et al.
Publicado: (2025)

CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models
por: Gong, Zi, et al.
Publicado: (2024)

Internalizing World Models via Self-Play Finetuning for Agentic RL
por: Chen, Shiqi, et al.
Publicado: (2025)

Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning
por: Ye, Jiasheng, et al.
Publicado: (2023)

ReFT: Representation Finetuning for Language Models
por: Wu, Zhengxuan, et al.
Publicado: (2024)

Vanishing Gradients in Reinforcement Finetuning of Language Models
por: Razin, Noam, et al.
Publicado: (2023)

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
por: Qin, Haotong, et al.
Publicado: (2024)

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
por: Longpre, Shayne, et al.
Publicado: (2025)

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
por: Liao, Baohao, et al.
Publicado: (2024)

MuonAll: Muon Variant for Efficient Finetuning of Large Language Models
por: Page, Saurabh, et al.
Publicado: (2025)

Orthogonal Finetuning Made Scalable
por: Qiu, Zeju, et al.
Publicado: (2025)

The Finetuner's Fallacy: When to Pretrain with Your Finetuning Data
por: Baek, Christina, et al.
Publicado: (2026)

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation
por: Halawi, Danny, et al.
Publicado: (2024)

Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
por: Shi, Taiwei, et al.
Publicado: (2025)

Assessing the Portability of Parameter Matrices Trained by Parameter-Efficient Finetuning Methods
por: Sabry, Mohammed, et al.
Publicado: (2024)

Orthogonal Finetuning for Direct Preference Optimization
por: Yang, Chenxu, et al.
Publicado: (2024)

PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
por: Hayou, Soufiane, et al.
Publicado: (2025)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
por: Feng, Jiahai, et al.
Publicado: (2024)

Representation Noising: A Defence Mechanism Against Harmful Finetuning
por: Rosati, Domenic, et al.
Publicado: (2024)

KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation
por: Luo, Xindi, et al.
Publicado: (2024)