Cover Image

Answer When Needed, Forget When Not: Language Models Pretend to Forget via In-Context Knowledge Unlearning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Takashiro, Shota, Kojima, Takeshi, Gambardella, Andrew, Cao, Qi, Iwasawa, Yusuke, Matsuo, Yutaka
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Inconsistent Tokenizations Cause Language Models to be Perplexed by Japanese Grammar
by: Gambardella, Andrew, et al.
Published: (2025)

Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
by: Cao, Qi, et al.
Published: (2026)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
by: Takashiro, Shota, et al.
Published: (2026)

Which Programming Language and What Features at Pre-training Stage Affect Downstream Logical Inference Performance?
by: Uchiyama, Fumiya, et al.
Published: (2024)

Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks
by: Gambardella, Andrew, et al.
Published: (2024)

Automated Refinement of Essay Scoring Rubrics for Language Models via Reflect-and-Revise
by: Harada, Keno, et al.
Published: (2025)

On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons
by: Kojima, Takeshi, et al.
Published: (2024)

Dynamic Injection of Entity Knowledge into Dense Retrievers
by: Yamada, Ikuya, et al.
Published: (2025)

When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Following
by: Harada, Keno, et al.
Published: (2025)

RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
by: Matsutani, Kohsei, et al.
Published: (2025)

When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?
by: Wang, Shang, et al.
Published: (2024)

Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflection
by: Yang, Bo, et al.
Published: (2025)

Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearning
by: Wan, Yixin, et al.
Published: (2025)

Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training
by: Matsutani, Kohsei, et al.
Published: (2026)

To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models
by: Tian, Bozhong, et al.
Published: (2024)

Safe Transformer: An Explicit Safety Bit For Interpretable And Controllable Alignment
by: Feng, Jingyuan, et al.
Published: (2026)

Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
by: Minegishi, Gouki, et al.
Published: (2025)

Scaling Laws for Forgetting When Fine-Tuning Large Language Models
by: Kalajdzievski, Damjan
Published: (2024)

Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
by: Shang, Bingqi, et al.
Published: (2025)

UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets
by: Wang, Wenyu, et al.
Published: (2025)

Thinking While Listening: Fast-Slow Recurrence for Long-Horizon Sequential Modeling
by: Takashiro, Shota, et al.
Published: (2026)

Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words
by: Minegishi, Gouki, et al.
Published: (2025)

CLIP-like Model as a Foundational Density Ratio Estimator
by: Uchiyama, Fumiya, et al.
Published: (2025)

Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models
by: Wang, Lingzhi, et al.
Published: (2024)

Self-Harmony: Learning to Harmonize Self-Supervision and Self-Play in Test-Time Reinforcement Learning
by: Wang, Ru, et al.
Published: (2025)

Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models
by: Cao, Boxi, et al.
Published: (2023)

FIT to Forget: Robust Continual Unlearning for Large Language Models
by: Xu, Xiaoyu, et al.
Published: (2026)

How to Make LLMs Forget: On Reversing In-Context Knowledge Edits
by: Youssef, Paul, et al.
Published: (2024)

Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness
by: Wei, Rongzhe, et al.
Published: (2025)

Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?
by: Tan, Hexiang, et al.
Published: (2024)

LLM Unlearning via Loss Adjustment with Only Forget Data
by: Wang, Yaxuan, et al.
Published: (2024)

Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution
by: Sadhu, Saisab, et al.
Published: (2026)

Beyond In-Distribution Success: Scaling Curves of CoT Granularity for Language Model Generalization
by: Wang, Ru, et al.
Published: (2025)

Understanding Emergent Misalignment via Feature Superposition Geometry
by: Minegishi, Gouki, et al.
Published: (2026)

Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models
by: Wang, Huazheng, et al.
Published: (2025)

Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties
by: Minegishi, Gouki, et al.
Published: (2025)

Investigating the Multilingual Calibration Effects of Language Model Instruction-Tuning
by: Huang, Jerry, et al.
Published: (2026)

LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
by: Dou, Shihan, et al.
Published: (2023)

Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models
by: Ni, Shiwen, et al.
Published: (2023)

Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities
by: Dang, Tien, et al.
Published: (2026)