Image de couverture de livre

Careful with that Scalpel: Improving Gradient Surgery with an EMA

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Hsieh, Yu-Guan, Thornton, James, Ndiaye, Eugene, Klein, Michal, Cuturi, Marco, Ablin, Pierre
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellency
par: Kirchhof, Michael, et autres
Publié: (2024)

Learning Elastic Costs to Shape Monge Displacements
par: Klein, Michal, et autres
Publié: (2023)

The Geometries of Truth Are Orthogonal Across Tasks
par: Azizian, Waiss, et autres
Publié: (2025)

Multivariate Conformal Prediction using Optimal Transport
par: Klein, Michal, et autres
Publié: (2025)

Nectar: Neural Estimation of Cached-Token Attention via Regression
par: Monteiro, João, et autres
Publié: (2026)

Contrasting Multiple Representations with the Multi-Marginal Matching Gap
par: Piran, Zoe, et autres
Publié: (2024)

Simple ReFlow: Improved Techniques for Fast Flow Models
par: Kim, Beomsu, et autres
Publié: (2024)

The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining
par: Saada, Thiziri Nait, et autres
Publié: (2025)

Locking Pretrained Weights via Deep Low-Rank Residual Distillation
par: Sakamoto, Keitaro, et autres
Publié: (2026)

Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration
par: Mlodozeniec, Bruno, et autres
Publié: (2025)

Progressive Entropic Optimal Transport Solvers
par: Kassraie, Parnian, et autres
Publié: (2024)

DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures
par: Gualdoni, Eleonora, et autres
Publié: (2026)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
par: Bethune, Louis, et autres
Publié: (2025)

On Fitting Flow Models with Large Sinkhorn Couplings
par: Zhang, Stephen, et autres
Publié: (2025)

Amortizing Maximum Inner Product Search with Learned Support Functions
par: Olausson, Theo X., et autres
Publié: (2026)

Scaling Categorical Flow Maps
par: Davis, Oscar, et autres
Publié: (2026)

Flow Matching with Semidiscrete Couplings
par: Mousavi-Hosseini, Alireza, et autres
Publié: (2025)

Dynamic Gradient Alignment for Online Data Mixing
par: Fan, Simin, et autres
Publié: (2024)

Omega: Optimistic EMA Gradients
par: Ramirez, Juan, et autres
Publié: (2023)

Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distribution to Multivariate Settings
par: Ndiaye, Eugene
Publié: (2025)

GENOT: Entropic (Gromov) Wasserstein Flow Matching with Applications to Single-Cell Genomics
par: Klein, Dominik, et autres
Publié: (2023)

Learning Unmasking Policies for Diffusion Language Models
par: Jazbec, Metod, et autres
Publié: (2025)

On a Neural Implementation of Brenier's Polar Factorization
par: Vesseron, Nina, et autres
Publié: (2024)

EMA Policy Gradient: Taming Reinforcement Learning for LLMs with EMA Anchor and Top-k KL
par: Zhang, Lunjun, et autres
Publié: (2026)

The AdEMAMix Optimizer: Better, Faster, Older
par: Pagliardini, Matteo, et autres
Publié: (2024)

How Smooth Is Attention?
par: Castin, Valérie, et autres
Publié: (2023)

Sample and Map from a Single Convex Potential: Generation using Conjugate Moment Measures
par: Vesseron, Nina, et autres
Publié: (2025)

From Conformal Predictions to Confidence Regions
par: Guille-Escuret, Charles, et autres
Publié: (2024)

Finite Sample Confidence Regions for Linear Regression Parameters Using Arbitrary Predictors
par: Guille-Escuret, Charles, et autres
Publié: (2024)

Exact and Approximate Conformal Inference for Multi-Output Regression
par: Johnstone, Chancellor, et autres
Publié: (2022)

Enhancing Hypergradients Estimation: A Study of Preconditioning and Reparameterization
par: Ye, Zhenzhang, et autres
Publié: (2024)

MVICAD2: Multi-View Independent Component Analysis with Delays and Dilations
par: Heurtebise, Ambroise, et autres
Publié: (2025)

GS-EMA: Integrating Gradient Surgery Exponential Moving Average with Boundary-Aware Contrastive Learning for Enhanced Domain Generalization in Aneurysm Segmentation
par: Lin, Fengming, et autres
Publié: (2024)

A Specialized Semismooth Newton Method for Kernel-Based Optimal Transport
par: Lin, Tianyi, et autres
Publié: (2023)

The Structural Scalpel: Automated Contiguous Layer Pruning for Large Language Models
par: Lu, Yao, et autres
Publié: (2025)

The Rogue Scalpel: Activation Steering Compromises LLM Safety
par: Korznikov, Anton, et autres
Publié: (2025)

Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling
par: Grangier, David, et autres
Publié: (2024)

Need a Small Specialized Language Model? Plan Early!
par: Grangier, David, et autres
Publié: (2024)

Scaling Laws for Mixture Pretraining Under Data Constraints
par: Sedova, Anastasiia, et autres
Publié: (2026)

A framework for bilevel optimization that enables stochastic and global variance reduction algorithms
par: Dagréou, Mathieu, et autres
Publié: (2022)