Cover Image

Subspace-based Approximate Hessian Method for Zeroth-Order Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Dongyoon, Lee, Sungjae, Lee, Wonjin, Kim, Kwang In
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Piece of Table: A Divide-and-Conquer Approach for Selecting Subtables in Table Question Answering
by: Lee, Wonjin, et al.
Published: (2024)

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
by: Yu, Ziming, et al.
Published: (2024)

SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation
by: Shin, Dahun, et al.
Published: (2025)

Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer
by: Feng, Jian, et al.
Published: (2026)

TAROT: Towards Essentially Domain-Invariant Robustness with Theoretical Justification
by: Yang, Dongyoon, et al.
Published: (2025)

A Historical Trajectory Assisted Optimization Method for Zeroth-Order Federated Learning
by: Wu, Chenlin, et al.
Published: (2024)

Refining Adaptive Zeroth-Order Optimization at Ease
by: Shu, Yao, et al.
Published: (2025)

Do's and Don'ts: Learning Desirable Skills with Instruction Videos
by: Kim, Hyunseung, et al.
Published: (2024)

HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
by: Zhao, Huaqin, et al.
Published: (2024)

Learning to Transfer Human Hand Skills for Robot Manipulations
by: Park, Sungjae, et al.
Published: (2025)

Causality-Aware Contrastive Learning for Robust Multivariate Time-Series Anomaly Detection
by: Kim, HyunGi, et al.
Published: (2025)

Zeroth-Order Optimization Finds Flat Minima
by: Zhang, Liang, et al.
Published: (2025)

GraspCorrect: Robotic Grasp Correction via Vision-Language Model-Guided Feedback
by: Lee, Sungjae, et al.
Published: (2025)

Alignment Tampering: How Reinforcement Learning from Human Feedback Is Exploited to Optimize Misaligned Biases
by: Hahm, Dongyoon, et al.
Published: (2026)

Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oracles
by: Tang, Zhiwei, et al.
Published: (2023)

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
by: Ma, Shaocong, et al.
Published: (2025)

NegMerge: Sign-Consensual Weight Merging for Machine Unlearning
by: Kim, Hyo Seo, et al.
Published: (2024)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
by: Guo, Wentao, et al.
Published: (2024)

Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning
by: Kim, Donghu, et al.
Published: (2024)

OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning
by: Long, Jikai, et al.
Published: (2025)

Second-Order Actor-Critic Methods for Discounted MDPs via Policy Hessian Decomposition
by: Manivannan, Sanjeev, et al.
Published: (2026)

Benchmarking Mobile Device Control Agents across Diverse Configurations
by: Lee, Juyong, et al.
Published: (2024)

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning
by: Liu, Z, et al.
Published: (2024)

MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models
by: Zhang, Zhen, et al.
Published: (2025)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
by: Wang, Shuo, et al.
Published: (2026)

Inversion-based Latent Bayesian Optimization
by: Chu, Jaewon, et al.
Published: (2024)

ICaRus: Identical Cache Reuse for Efficient Multi Model Inference
by: Woo, Sunghyeon, et al.
Published: (2026)

Elucidating Subspace Perturbation in Zeroth-Order Optimization: Theory and Practice at Scale
by: Park, Sihwan, et al.
Published: (2025)

Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens
by: Qiu, Junbin, et al.
Published: (2026)

FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed
by: Dang, Sizhe, et al.
Published: (2025)

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
by: Cai, Zhijie, et al.
Published: (2026)

TRACED: Transition-aware Regret Approximation with Co-learnability for Environment Design
by: Cho, Geonwoo, et al.
Published: (2025)

Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning
by: Elsayed, Mohamed, et al.
Published: (2024)

Advanced Torrential Loss Function for Precipitation Forecasting
by: Choi, Jaeho, et al.
Published: (2025)

Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization
by: Liang, Chen, et al.
Published: (2026)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
by: Gautam, Tanmay, et al.
Published: (2024)

Warming Up for Zeroth-Order Federated Pre-Training with Low Resource Clients
by: Legate, Gwen, et al.
Published: (2025)

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
by: Zhou, Jiajun, et al.
Published: (2025)

SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety
by: Kim, Geon-Hyeong, et al.
Published: (2025)

Riemannian Optimization for LoRA on the Stiefel Manifold
by: Park, Juneyoung, et al.
Published: (2025)