Cover Image

Optimizing ML Training with Metagradient Descent

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Engstrom, Logan, Ilyas, Andrew, Chen, Benjamin, Feldmann, Axel, Moses, William, Madry, Aleksander
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DsDm: Model-Aware Dataset Selection with Datamodels
by: Engstrom, Logan, et al.
Published: (2024)

Decomposing and Editing Predictions by Modeling Model Computation
by: Shah, Harshay, et al.
Published: (2024)

Optimizing Canaries for Privacy Auditing with Metagradient Descent
by: Boglioni, Matteo, et al.
Published: (2025)

Small-to-Large Generalization: Data Influences Models Consistently Across Scale
by: Khaddaj, Alaa, et al.
Published: (2025)

DataMIL: Selecting Data for Robot Imitation Learning with Datamodels
by: Dass, Shivin, et al.
Published: (2025)

MAGIC: Near-Optimal Data Attribution for Deep Learning
by: Ilyas, Andrew, et al.
Published: (2025)

Auto-Unrolled Proximal Gradient Descent: An AutoML Approach to Interpretable Waveform Optimization
by: Kaplan, Ahmet
Published: (2026)

User Strategization and Trustworthy Algorithms
by: Cen, Sarah H., et al.
Published: (2023)

Ask Your Distribution Shift if Pre-Training is Right for You
by: Cohen-Wang, Benjamin, et al.
Published: (2024)

A Continual and Incremental Learning Approach for TinyML On-device Training Using Dataset Distillation and Model Size Adaption
by: Rüb, Marcus, et al.
Published: (2024)

Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Training
by: Ma, Yuhan, et al.
Published: (2024)

Data Pipeline Training: Integrating AutoML to Optimize the Data Flow of Machine Learning Models
by: Wu, Jiang, et al.
Published: (2024)

Memory-Efficient LLM Training with Online Subspace Descent
by: Liang, Kaizhao, et al.
Published: (2024)

Revisiting the Initial Steps in Adaptive Gradient Descent Optimization
by: Abuduweili, Abulikemu, et al.
Published: (2024)

Exploiting Block Coordinate Descent for Cost-Effective LLM Model Training
by: Liu, Zeyu, et al.
Published: (2025)

Local Entropy Search over Descent Sequences for Bayesian Optimization
by: Stenger, David, et al.
Published: (2025)

Activation-Descent Regularization for Input Optimization of ReLU Networks
by: Yu, Hongzhan, et al.
Published: (2024)

Counterfactual Training: Teaching Models Plausible and Actionable Explanations
by: Altmeyer, Patrick, et al.
Published: (2026)

Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization
by: Kumar, Ramnath, et al.
Published: (2023)

Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training
by: Lu, Yishun, et al.
Published: (2025)

Jacobian Descent for Multi-Objective Optimization
by: Quinton, Pierre, et al.
Published: (2024)

In-Context Decision Making for Optimizing Complex AutoML Pipelines
by: Balef, Amir Rezaei, et al.
Published: (2025)

Beyond State-Wise Mirror Descent: Offline Policy Optimization with Parametric Policies
by: Li, Xiang, et al.
Published: (2026)

PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization
by: Xu, Mingjing, et al.
Published: (2024)

Optimization, Generalization and Differential Privacy Bounds for Gradient Descent on Kolmogorov-Arnold Networks
by: Wang, Puyu, et al.
Published: (2026)

Reconstructing Deep Neural Networks: Unleashing the Optimization Potential of Natural Gradient Descent
by: Liu, Weihua, et al.
Published: (2024)

Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization
by: Liang, Chen, et al.
Published: (2026)

EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environments
by: Mehta, Sushant, et al.
Published: (2026)

Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection
by: Jain, Saachi, et al.
Published: (2024)

The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization
by: Chung, Jae-Won, et al.
Published: (2025)

Preference-Based Gradient Estimation for ML-Guided Approximate Combinatorial Optimization
by: Mielke, Arman, et al.
Published: (2025)

Combining Multi-Objective Bayesian Optimization with Reinforcement Learning for TinyML
by: Deutel, Mark, et al.
Published: (2023)

Optimizing Predictive AI in Physical Design Flows with Mini Pixel Batch Gradient Descent
by: Yang, Haoyu, et al.
Published: (2024)

ML For Hardware Design Interpretability: Challenges and Opportunities
by: Baartmans, Raymond, et al.
Published: (2025)

A Universal Banach--Bregman Framework for Stochastic Iterations: Unifying Stochastic Mirror Descent, Learning and LLM Training
by: Zhang, Johnny R., et al.
Published: (2025)

Automated Computational Energy Minimization of ML Algorithms using Constrained Bayesian Optimization
by: Mitra, Pallavi, et al.
Published: (2024)

Gradient Descent Algorithm Survey
by: Fucheng, Deng, et al.
Published: (2025)

Policy Mirror Descent with Lookahead
by: Protopapas, Kimon, et al.
Published: (2024)

ROOT: Robust Orthogonalized Optimizer for Neural Network Training
by: He, Wei, et al.
Published: (2025)

ML-Tool-Bench: Tool-Augmented Planning for ML Tasks
by: Chittepu, Yaswanth, et al.
Published: (2025)