Image de couverture de livre

Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liu, Sijia, Lang, Yicheng, Pal, Soumyadeep, Wang, Changsheng, Huang, Yancheng, Fan, Chongyu, Diffenderfer, James, Kailkhura, Bhavya, Zhang, Yihua
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks
par: Pal, Soumyadeep, et autres
Publié: (2025)

DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training
par: Chen, Aochuan, et autres
Publié: (2023)

LLM Unlearning Under the Microscope: A Full-Stack View on Methods and Metrics
par: Fan, Chongyu, et autres
Publié: (2025)

SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
par: Jia, Jinghan, et autres
Publié: (2024)

Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning
par: Lang, Yicheng, et autres
Publié: (2025)

End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver
par: Ma, Shaocong, et autres
Publié: (2024)

Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
par: Lang, Yicheng, et autres
Publié: (2026)

Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization
par: Huang, Yancheng, et autres
Publié: (2026)

Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies
par: Bartoldson, Brian R., et autres
Publié: (2024)

Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models
par: Arif, Huzaifa, et autres
Publié: (2025)

UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making
par: Duan, Jinhao, et autres
Publié: (2025)

Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills
par: Wang, Changsheng, et autres
Publié: (2025)

Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning
par: Wang, Changsheng, et autres
Publié: (2025)

SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
par: Fan, Chongyu, et autres
Publié: (2023)

Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
par: Fan, Chongyu, et autres
Publié: (2025)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
par: Zhang, Yihua, et autres
Publié: (2024)

Backdoor Secrets Unveiled: Identifying Backdoor Data with Optimized Scaled Prediction Consistency
par: Pal, Soumyadeep, et autres
Publié: (2024)

Certifiably-Robust Federated Adversarial Learning via Randomized Smoothing
par: Chen, Cheng, et autres
Publié: (2021)

Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training
par: Bartoldson, Brian, et autres
Publié: (2025)

FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling
par: Chen, Cheng, et autres
Publié: (2020)

Learning Dynamics of Zeroth-Order Optimization: A Kernel Perspective
par: Li, Zhe, et autres
Publié: (2026)

GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
par: Duan, Jinhao, et autres
Publié: (2024)

Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs
par: Chen, Yiwei, et autres
Publié: (2025)

On Adaptivity in Zeroth-Order Optimization
par: Dbouk, Hassan, et autres
Publié: (2026)

Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning
par: Fan, Chongyu, et autres
Publié: (2024)

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
par: Zhang, Kairun, et autres
Publié: (2025)

Private Zeroth-Order Optimization with Public Data
par: Gong, Xuchen, et autres
Publié: (2025)

The Multi-Query Paradox in Zeroth-Order Optimization
par: Lin, Wei, et autres
Publié: (2025)

Zeroth-Order Methods for Stochastic Nonconvex Nonsmooth Composite Optimization
par: Chen, Ziyi, et autres
Publié: (2025)

Improving Robustness In Sparse Autoencoders via Masked Regularization
par: Narayanaswamy, Vivek, et autres
Publié: (2026)

Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
par: Yang, Hongru, et autres
Publié: (2024)

Zeroth-Order Optimization at the Edge of Stability
par: Song, Minhak, et autres
Publié: (2026)

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
par: Ma, Shaocong, et autres
Publié: (2025)

Leak@$k$: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding
par: Reisizadeh, Hadi, et autres
Publié: (2025)

Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR
par: Fan, Chongyu, et autres
Publié: (2026)

Refining Adaptive Zeroth-Order Optimization at Ease
par: Shu, Yao, et autres
Publié: (2025)

Zeroth-Order Optimization is Secretly Single-Step Policy Optimization
par: Qiu, Junbin, et autres
Publié: (2025)

CyclicReflex: Improving Reasoning Models via Cyclical Reflection Token Scheduling
par: Fan, Chongyu, et autres
Publié: (2025)

Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
par: Fan, Chongyu, et autres
Publié: (2024)

Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization
par: Li, Zhe, et autres
Publié: (2024)