Cover Image

Vanishing Gradients in Reinforcement Finetuning of Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Razin, Noam, Zhou, Hattie, Saremi, Omid, Thilak, Vimal, Bradley, Arwen, Nakkiran, Preetum, Susskind, Joshua, Littwin, Etai
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks
by: Littwin, Etai, et al.
Published: (2024)

Step-by-Step Diffusion: An Elementary Tutorial
by: Nakkiran, Preetum, et al.
Published: (2024)

Classifier-Free Guidance is a Predictor-Corrector
by: Bradley, Arwen, et al.
Published: (2024)

When can transformers reason with abstract symbols?
by: Boix-Adsera, Enric, et al.
Published: (2023)

Annotations Mitigate Post-Training Mode Collapse
by: Springer, Jacob Mitchell, et al.
Published: (2026)

Trace Length is a Simple Uncertainty Signal in Reasoning Models
by: Devic, Siddartha, et al.
Published: (2025)

Enhancing JEPAs with Spatial Conditioning: Robust and Efficient Representation Learning
by: Littwin, Etai, et al.
Published: (2024)

Text-Conditional JEPA for Learning Semantically Rich Visual Representations
by: Huang, Chen, et al.
Published: (2026)

To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models
by: Malach, Eran, et al.
Published: (2025)

Mechanisms of Projective Composition of Diffusion Models
by: Bradley, Arwen, et al.
Published: (2025)

Trained on Tokens, Calibrated on Concepts: The Emergence of Semantic Calibration in LLMs
by: Nakkiran, Preetum, et al.
Published: (2025)

Why is Your Language Model a Poor Implicit Reward Model?
by: Razin, Noam, et al.
Published: (2025)

UI-JEPA: Towards Active Perception of User Intent through Onscreen User Activity
by: Fu, Yicheng, et al.
Published: (2024)

Distillation Scaling Laws
by: Busbridge, Dan, et al.
Published: (2025)

Rethinking JEPA: Compute-Efficient Video SSL with Frozen Teachers
by: Li, Xianhang, et al.
Published: (2025)

Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models
by: Abnar, Samira, et al.
Published: (2025)

Understanding Deep Learning via Notions of Rank
by: Razin, Noam
Published: (2024)

Contradiction Detection in RAG Systems: Evaluating LLMs as Context Validators for Improved Information Consistency
by: Gokul, Vignesh, et al.
Published: (2025)

What Makes a Reward Model a Good Teacher? An Optimization Perspective
by: Razin, Noam, et al.
Published: (2025)

Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning
by: Cohen, Nadav, et al.
Published: (2024)

Risk-Averse Finetuning of Large Language Models
by: Chaudhary, Sapana, et al.
Published: (2025)

Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlo
by: Thornton, James, et al.
Published: (2025)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
by: Razin, Noam, et al.
Published: (2024)

Whisper Finetuning on Nepali Language
by: Rijal, Sanjay, et al.
Published: (2024)

Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model
by: Liang, Jing, et al.
Published: (2025)

CARE-RFT: Confidence-Anchored Reinforcement Finetuning for Reliable Reasoning in Large Language Models
by: Li, Shuozhe, et al.
Published: (2026)

A Simple "Motivation" Can Enhance Reinforcement Finetuning of Large Reasoning Models
by: Zhang, Junjie, et al.
Published: (2025)

Large Language Models to Diffusion Finetuning
by: Cetin, Edoardo, et al.
Published: (2025)

Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization
by: Zang, Yuhang, et al.
Published: (2024)

ReFT: Representation Finetuning for Language Models
by: Wu, Zhengxuan, et al.
Published: (2024)

LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models
by: Stengel-Eskin, Elias, et al.
Published: (2024)

Aligning Large Language Models with Counterfactual DPO
by: Butcher, Bradley
Published: (2024)

When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient
by: Shang, Shuning, et al.
Published: (2026)

Finetuning Generative Large Language Models with Discrimination Instructions for Knowledge Graph Completion
by: Liu, Yang, et al.
Published: (2024)

Zero-Shot Classification of Crisis Tweets Using Instruction-Finetuned Large Language Models
by: McDaniel, Emma, et al.
Published: (2024)

On Instruction-Finetuning Neural Machine Translation Models
by: Raunak, Vikas, et al.
Published: (2024)

A Tutorial on the Pretrain-Finetune Paradigm for Natural Language Processing
by: Wang, Yu, et al.
Published: (2024)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
by: Kang, Katie, et al.
Published: (2024)

Transfer Learning of Tabular Data by Finetuning Large Language Models
by: Rabbani, Shourav B., et al.
Published: (2025)

Fundamental Limitations of Alignment in Large Language Models
by: Wolf, Yotam, et al.
Published: (2023)