Cover Image

Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bu, Zhiqi, Jin, Xiaomeng, Vinzamuri, Bhanukiran, Ramakrishna, Anil, Chang, Kai-Wei, Cevher, Volkan, Hong, Mingyi
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LUME: LLM Unlearning with Multitask Evaluations
by: Ramakrishna, Anil, et al.
Published: (2025)

SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models
by: Ramakrishna, Anil, et al.
Published: (2025)

BLUR: A Bi-Level Optimization Approach for LLM Unlearning
by: Reisizadeh, Hadi, et al.
Published: (2025)

Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearning
by: Wan, Yixin, et al.
Published: (2025)

MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
by: Ozkara, Kaan, et al.
Published: (2024)

Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
by: Fan, Chongyu, et al.
Published: (2025)

Easy Data Unlearning Bench
by: Rinberg, Roy, et al.
Published: (2026)

A Hessian-informed hyperparameter optimization for differential learning rate
by: Xu, Shiyun, et al.
Published: (2025)

DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction
by: Zhang, Xinwei, et al.
Published: (2024)

Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
by: Jiang, Ruichen, et al.
Published: (2024)

Rate optimal learning of equilibria from data
by: Freihaut, Till, et al.
Published: (2025)

Adaptive multi-gradient methods for quasiconvex vector optimization and applications to multi-task learning
by: Minh, Nguyen Anh, et al.
Published: (2024)

Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach
by: Zhang, Xinwei, et al.
Published: (2023)

Advancing the lower bounds: An accelerated, stochastic, second-order method with optimal adaptation to inexactness
by: Agafonov, Artem, et al.
Published: (2023)

Provably avoiding over-optimization in Direct Preference Optimization without knowing the data distribution
by: Barla, Adam, et al.
Published: (2026)

Pre-training Differentially Private Models with Limited Public Data
by: Bu, Zhiqi, et al.
Published: (2024)

Towards hyperparameter-free optimization with differential privacy
by: Bu, Zhiqi, et al.
Published: (2025)

From Narrow Unlearning to Emergent Misalignment: Causes, Consequences, and Containment in LLMs
by: Mushtaq, Erum, et al.
Published: (2025)

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
by: Xie, Wanyun, et al.
Published: (2025)

Efficient Large Language Model Inference with Neural Block Linearization
by: Erdogan, Mete, et al.
Published: (2025)

Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks
by: Liu, Fanghui, et al.
Published: (2024)

SAMPa: Sharpness-aware Minimization Parallelized
by: Xie, Wanyun, et al.
Published: (2024)

MT-NAM: An Efficient and Adaptive Model for Epileptic Seizure Detection
by: Afzal, Arshia, et al.
Published: (2025)

Matching Multiple Experts: On the Exploitability of Multi-Agent Imitation Learning
by: Bergerault, Antoine, et al.
Published: (2026)

MaD-Mix: Multi-Modal Data Mixtures via Latent Space Coupling for Vision-Language Model Training
by: Xie, Wanyun, et al.
Published: (2026)

Stable Nonconvex-Nonconcave Training via Linear Interpolation
by: Pethick, Thomas, et al.
Published: (2023)

IL-SOAR : Imitation Learning with Soft Optimistic Actor cRitic
by: Viel, Stefano, et al.
Published: (2025)

Imitation Learning in Discounted Linear MDPs without exploration assumptions
by: Viano, Luca, et al.
Published: (2024)

Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models
by: Entesari, Taha, et al.
Published: (2025)

DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction
by: Zhang, Xinwei, et al.
Published: (2024)

μP$^2$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
by: Haas, Moritz, et al.
Published: (2024)

Hadamard product in deep learning: Introduction, Advances and Challenges
by: Chrysos, Grigorios G, et al.
Published: (2025)

Scaling depth capacity via zero/one-layer model expansion
by: Bu, Zhiqi
Published: (2025)

Task-conditioned adaptation of visual features in multi-task policy learning
by: Marza, Pierre, et al.
Published: (2024)

Adversarial Training for Defense Against Label Poisoning Attacks
by: Bal, Melis Ilayda, et al.
Published: (2025)

ESLM: Risk-Averse Selective Language Modeling for Efficient Pretraining
by: Bal, Melis Ilayda, et al.
Published: (2025)

Randomized algorithms and PAC bounds for inverse reinforcement learning in continuous spaces
by: Kamoutsi, Angeliki, et al.
Published: (2024)

Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates
by: Dereich, Steffen, et al.
Published: (2024)

Multi-agent imitation learning with function approximation: Linear Markov games and beyond
by: Viano, Luca, et al.
Published: (2026)

Learning to Remove Cuts in Integer Linear Programming
by: Puigdemont, Pol, et al.
Published: (2024)