Cover Image

DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Aochuan, Zhang, Yimeng, Jia, Jinghan, Diffenderfer, James, Liu, Jiancheng, Parasyris, Konstantinos, Zhang, Yihua, Zhang, Zheng, Kailkhura, Bhavya, Liu, Sijia
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
by: Jia, Jinghan, et al.
Published: (2024)

Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
by: Liu, Sijia, et al.
Published: (2026)

Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies
by: Bartoldson, Brian R., et al.
Published: (2024)

To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now
by: Zhang, Yimeng, et al.
Published: (2023)

End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver
by: Ma, Shaocong, et al.
Published: (2024)

LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks
by: Pal, Soumyadeep, et al.
Published: (2025)

Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
by: Lang, Yicheng, et al.
Published: (2026)

Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models
by: Zhang, Yimeng, et al.
Published: (2024)

WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models
by: Jia, Jinghan, et al.
Published: (2024)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
by: Zhang, Yihua, et al.
Published: (2024)

Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models
by: Arif, Huzaifa, et al.
Published: (2025)

The Power of Few: Accelerating and Enhancing Data Reweighting with Coreset Selection
by: Jafari, Mohammad, et al.
Published: (2024)

UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Models
by: Zhang, Yihua, et al.
Published: (2024)

Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning
by: Lang, Yicheng, et al.
Published: (2025)

Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classification
by: Sun, Changchang, et al.
Published: (2024)

UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making
by: Duan, Jinhao, et al.
Published: (2025)

CyclicReflex: Improving Reasoning Models via Cyclical Reflection Token Scheduling
by: Fan, Chongyu, et al.
Published: (2025)

Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
by: Fan, Chongyu, et al.
Published: (2024)

One Token Embedding Is Enough to Deadlock Your Large Reasoning Model
by: Zhang, Mohan, et al.
Published: (2025)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
by: Zhuang, Haomin, et al.
Published: (2024)

Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
by: Fan, Chongyu, et al.
Published: (2025)

LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Data
by: Wang, Changsheng, et al.
Published: (2025)

Visual prompting reimagined: The power of the Activation Prompts
by: Zhang, Yihua, et al.
Published: (2026)

Pruning then Reweighting: Towards Data-Efficient Training of Diffusion Models
by: Li, Yize, et al.
Published: (2024)

SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
by: Fan, Chongyu, et al.
Published: (2023)

GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
by: Duan, Jinhao, et al.
Published: (2024)

Nonlinear-Gain Distributed Zeroth-Order Optimization for Networked Black-Box Control
by: Zhang, Shengjun, et al.
Published: (2026)

Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training
by: Bartoldson, Brian, et al.
Published: (2025)

Private Zeroth-Order Nonsmooth Nonconvex Optimization
by: Zhang, Qinzi, et al.
Published: (2024)

Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
by: Yang, Hongru, et al.
Published: (2024)

Zeroth-Order Optimization at the Edge of Stability
by: Song, Minhak, et al.
Published: (2026)

Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability
by: Jia, Jinghan, et al.
Published: (2025)

Refining Adaptive Zeroth-Order Optimization at Ease
by: Shu, Yao, et al.
Published: (2025)

TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention
by: Duan, Jinhao, et al.
Published: (2025)

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
by: Zhang, Kairun, et al.
Published: (2025)

Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills
by: Wang, Changsheng, et al.
Published: (2025)

Zeroth-Order Optimization Finds Flat Minima
by: Zhang, Liang, et al.
Published: (2025)

Dynamic Weight Adjusting Deep Q-Networks for Real-Time Environmental Adaptation
by: Zhang, Xinhao, et al.
Published: (2024)

A Doubly Stochastically Perturbed Algorithm for Linearly Constrained Bilevel Optimization
by: Khanduri, Prashant, et al.
Published: (2025)

Efficient Federated RLHF via Zeroth-Order Policy Optimization
by: Wang, Deyi, et al.
Published: (2026)