Imagen de Portada

Predicting Emergent Capabilities by Finetuning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Snell, Charlie, Wallace, Eric, Klein, Dan, Levine, Sergey
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
por: Kang, Katie, et al.
Publicado: (2024)

What Evidence Do Language Models Find Convincing?
por: Wan, Alexander, et al.
Publicado: (2024)

Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
por: Snell, Charlie, et al.
Publicado: (2024)

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation
por: Halawi, Danny, et al.
Publicado: (2024)

From Style to Facts: Mapping the Boundaries of Knowledge Injection with Finetuning
por: Zhao, Eric, et al.
Publicado: (2025)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
por: Bethune, Louis, et al.
Publicado: (2025)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
por: Zhang, Biao, et al.
Publicado: (2024)

e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs
por: Setlur, Amrith, et al.
Publicado: (2025)

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
por: Gupta, Prakhar, et al.
Publicado: (2026)

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
por: Xiong, Zheyang, et al.
Publicado: (2024)

Value-Based Deep RL Scales Predictably
por: Rybkin, Oleh, et al.
Publicado: (2025)

Scaling Test-Time Compute Without Verification or RL is Suboptimal
por: Setlur, Amrith, et al.
Publicado: (2025)

Improving Sparse Memory Finetuning
por: Goyal, Satyam, et al.
Publicado: (2026)

ReFT: Representation Finetuning for Language Models
por: Wu, Zhengxuan, et al.
Publicado: (2024)

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
por: Guo, Han, et al.
Publicado: (2023)

Sparse Upcycling: Inference Inefficient Finetuning
por: Doubov, Sasha, et al.
Publicado: (2024)

Understanding Finetuning for Factual Knowledge Extraction
por: Ghosal, Gaurav, et al.
Publicado: (2024)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
por: Hong, Joey, et al.
Publicado: (2024)

Task Matrices: Linear Maps for Cross-Model Finetuning Transfer
por: Brien, Darrin O', et al.
Publicado: (2025)

Learning to Reason without External Rewards
por: Zhao, Xuandong, et al.
Publicado: (2025)

Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space
por: Hong, Joey, et al.
Publicado: (2025)

Ensembling Finetuned Language Models for Text Classification
por: Arango, Sebastian Pineda, et al.
Publicado: (2024)

Transfer Learning for Finetuning Large Language Models
por: Strangmann, Tobias, et al.
Publicado: (2024)

Finetuning Language Models to Emit Linguistic Expressions of Uncertainty
por: Chaudhry, Arslan, et al.
Publicado: (2024)

Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
por: Shi, Taiwei, et al.
Publicado: (2025)

Order Independence With Finetuning
por: Brown, Katrina, et al.
Publicado: (2025)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
por: Feng, Jiahai, et al.
Publicado: (2024)

Representation Noising: A Defence Mechanism Against Harmful Finetuning
por: Rosati, Domenic, et al.
Publicado: (2024)

KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation
por: Luo, Xindi, et al.
Publicado: (2024)

Towards Active Synthetic Data Generation for Finetuning Language Models
por: Kessler, Samuel, et al.
Publicado: (2025)

Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations
por: Hong, Joey, et al.
Publicado: (2024)

Learning Dynamics of LLM Finetuning
por: Ren, Yi, et al.
Publicado: (2024)

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
por: Liao, Baohao, et al.
Publicado: (2024)

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
por: Qin, Haotong, et al.
Publicado: (2024)

CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models
por: Gong, Zi, et al.
Publicado: (2024)

Internalizing World Models via Self-Play Finetuning for Agentic RL
por: Chen, Shiqi, et al.
Publicado: (2025)

MuonAll: Muon Variant for Efficient Finetuning of Large Language Models
por: Page, Saurabh, et al.
Publicado: (2025)

OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
por: Feng, Jinyuan, et al.
Publicado: (2025)

Mitigating Catastrophic Forgetting in Mathematical Reasoning Finetuning through Mixed Training
por: Reynolds, John Graham
Publicado: (2025)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
por: Goyal, Sachin, et al.
Publicado: (2024)