Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914568408662016 |
|---|---|
| author | Dang, Tien Nguyen, The-Hai Phuong, Dinh Mai Phuong, Nguyen Minh Bui, Anh Thanh-Tung, Hoang Nguyen, Le-Minh Inoue, Naoya |
| author_facet | Dang, Tien Nguyen, The-Hai Phuong, Dinh Mai Phuong, Nguyen Minh Bui, Anh Thanh-Tung, Hoang Nguyen, Le-Minh Inoue, Naoya |
| contents | We consider Representation Misdirection (RM), a class of large language model (LLM) unlearning methods that achieve forgetting by redirecting the forget-representations, that is, latent representations of forget-samples, toward a target vector. Despite being important, the roles of the target vector used in RM, however, remain underexplored. Here, we approach and revisit RM through the lens of the Linear Representation Hypothesis. Specifically, if one can identify a one-dimensional representation corresponding to a high-level concept, the Linear Representation Hypothesis enables linear operations on this concept vector within the forget-representation space. Under this view, we hypothesize that, beyond forgetting, machine unlearning via RM elicits controllable emergent side behaviors and stronger side capabilities corresponding to the high-level concept. Our hypothesis is empirically validated across a wide range of tasks, including behavioral control (e.g., controlling unlearned models' truthfulness, sentiment, refusal, and language) and capability enhancement (e.g., improving unlearned models' in-context learning (ICL) capability). Our findings reveal that this phenomenon could be either a hidden risk if misused or a mechanism that can be harnessed for developing unlearned models that require stronger capabilities and controllable behaviors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_21702 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities Dang, Tien Nguyen, The-Hai Phuong, Dinh Mai Phuong, Nguyen Minh Bui, Anh Thanh-Tung, Hoang Nguyen, Le-Minh Inoue, Naoya Machine Learning Computation and Language We consider Representation Misdirection (RM), a class of large language model (LLM) unlearning methods that achieve forgetting by redirecting the forget-representations, that is, latent representations of forget-samples, toward a target vector. Despite being important, the roles of the target vector used in RM, however, remain underexplored. Here, we approach and revisit RM through the lens of the Linear Representation Hypothesis. Specifically, if one can identify a one-dimensional representation corresponding to a high-level concept, the Linear Representation Hypothesis enables linear operations on this concept vector within the forget-representation space. Under this view, we hypothesize that, beyond forgetting, machine unlearning via RM elicits controllable emergent side behaviors and stronger side capabilities corresponding to the high-level concept. Our hypothesis is empirically validated across a wide range of tasks, including behavioral control (e.g., controlling unlearned models' truthfulness, sentiment, refusal, and language) and capability enhancement (e.g., improving unlearned models' in-context learning (ICL) capability). Our findings reveal that this phenomenon could be either a hidden risk if misused or a mechanism that can be harnessed for developing unlearned models that require stronger capabilities and controllable behaviors. |
| title | Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2601.21702 |