Imagen de Portada

SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Jia, Jinghan, Zhang, Yihua, Zhang, Yimeng, Liu, Jiancheng, Runwal, Bharat, Diffenderfer, James, Kailkhura, Bhavya, Liu, Sijia
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training
por: Chen, Aochuan, et al.
Publicado: (2023)

LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks
por: Pal, Soumyadeep, et al.
Publicado: (2025)

Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
por: Fan, Chongyu, et al.
Publicado: (2024)

Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
por: Liu, Sijia, et al.
Publicado: (2026)

Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
por: Fan, Chongyu, et al.
Publicado: (2025)

WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models
por: Jia, Jinghan, et al.
Publicado: (2024)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
por: Zhuang, Haomin, et al.
Publicado: (2024)

Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning
por: Lang, Yicheng, et al.
Publicado: (2025)

Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
por: Shang, Bingqi, et al.
Publicado: (2025)

End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver
por: Ma, Shaocong, et al.
Publicado: (2024)

UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making
por: Duan, Jinhao, et al.
Publicado: (2025)

Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classification
por: Sun, Changchang, et al.
Publicado: (2024)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
por: Zhang, Yihua, et al.
Publicado: (2024)

The Power of Few: Accelerating and Enhancing Data Reweighting with Coreset Selection
por: Jafari, Mohammad, et al.
Publicado: (2024)

LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Data
por: Wang, Changsheng, et al.
Publicado: (2025)

Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies
por: Bartoldson, Brian R., et al.
Publicado: (2024)

GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
por: Duan, Jinhao, et al.
Publicado: (2024)

Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models
por: Arif, Huzaifa, et al.
Publicado: (2025)

Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning
por: Wang, Changsheng, et al.
Publicado: (2025)

Model Sparsity Can Simplify Machine Unlearning
por: Jia, Jinghan, et al.
Publicado: (2023)

Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning
por: Gu, Renjie, et al.
Publicado: (2026)

Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
por: Yang, Hongru, et al.
Publicado: (2024)

SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
por: Fan, Chongyu, et al.
Publicado: (2023)

Label Smoothing Improves Machine Unlearning
por: Di, Zonglin, et al.
Publicado: (2024)

Rethinking Machine Unlearning for Large Language Models
por: Liu, Sijia, et al.
Publicado: (2024)

LLM Unlearning Under the Microscope: A Full-Stack View on Methods and Metrics
por: Fan, Chongyu, et al.
Publicado: (2025)

Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills
por: Wang, Changsheng, et al.
Publicado: (2025)

Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs
por: Chen, Yiwei, et al.
Publicado: (2025)

BLUR: A Bi-Level Optimization Approach for LLM Unlearning
por: Reisizadeh, Hadi, et al.
Publicado: (2025)

Constrained Discrete Diffusion
por: Cardei, Michael, et al.
Publicado: (2025)

LLM Unlearning Without an Expert Curated Dataset
por: Zhu, Xiaoyuan, et al.
Publicado: (2025)

To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now
por: Zhang, Yimeng, et al.
Publicado: (2023)

Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
por: Lang, Yicheng, et al.
Publicado: (2026)

LLM Unlearning with LLM Beliefs
por: Li, Kemou, et al.
Publicado: (2025)

One Token Embedding Is Enough to Deadlock Your Large Reasoning Model
por: Zhang, Mohan, et al.
Publicado: (2025)

Attention Smoothing Is All You Need For Unlearning
por: Zade, Saleh Zare, et al.
Publicado: (2026)

Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models
por: Zhang, Yimeng, et al.
Publicado: (2024)

Efficient Second-Order Neural Network Optimization via Adaptive Trust Region Methods
por: Vo, James
Publicado: (2024)

From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers
por: Runwal, Bharat, et al.
Publicado: (2024)

Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training
por: Bartoldson, Brian, et al.
Publicado: (2025)