Cover Image

Adaptive Computation Pruning for the Forgetting Transformer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lin, Zhixuan, Obando-Ceron, Johan, He, Xu Owen, Courville, Aaron
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Forgetting Transformer: Softmax Attention with a Forget Gate
by: Lin, Zhixuan, et al.
Published: (2025)

In value-based deep reinforcement learning, a pruned network is a good network
by: Obando-Ceron, Johan, et al.
Published: (2024)

The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networks
by: Mayor, Walter, et al.
Published: (2025)

Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn
by: Tang, Hongyao, et al.
Published: (2025)

On the consistency of hyper-parameter selection in value-based deep reinforcement learning
by: Obando-Ceron, Johan, et al.
Published: (2024)

Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RL
by: Sokar, Ghada, et al.
Published: (2024)

Stable Deep Reinforcement Learning via Isotropic Gaussian Representations
by: Pasand, Ali Saheb, et al.
Published: (2026)

Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study
by: Tan, Shawn, et al.
Published: (2024)

Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents
by: Obando-Ceron, Johan, et al.
Published: (2025)

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting
by: Diao, Muxi, et al.
Published: (2026)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
by: Noukhovitch, Michael, et al.
Published: (2024)

V-STaR: Training Verifiers for Self-Taught Reasoners
by: Hosseini, Arian, et al.
Published: (2024)

Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
by: Kenneweg, Philip, et al.
Published: (2024)

A Mechanistic Analysis of Looped Reasoning Language Models
by: Blayney, Hugh, et al.
Published: (2026)

The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
by: Aghajohari, Milad, et al.
Published: (2025)

KVzap: Fast, Adaptive, and Faithful KV Cache Pruning
by: Jegou, Simon, et al.
Published: (2026)

Sample-aware Adaptive Structured Pruning for Large Language Models
by: Kong, Jun, et al.
Published: (2025)

Adaptive Pruning for Large Language Models with Structural Importance Awareness
by: Zheng, Haotian, et al.
Published: (2024)

Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation
by: Dong, Harry, et al.
Published: (2024)

Unleashing Diverse Thinking Modes in LLMs through Multi-Agent Collaboration
by: He, Zhixuan, et al.
Published: (2025)

Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference
by: Taniguchi, Rei, et al.
Published: (2026)

Do Efficient Transformers Really Save Computation?
by: Yang, Kai, et al.
Published: (2024)

Stuffed Mamba: Oversized States Lead to the Inability to Forget
by: Chen, Yingfa, et al.
Published: (2024)

Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment
by: Liu, Jun, et al.
Published: (2024)

Adapt-Pruner: Adaptive Structural Pruning for Efficient Small Language Model Training
by: Pan, Rui, et al.
Published: (2025)

Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training
by: Cunegatti, Elia, et al.
Published: (2024)

ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization
by: Shopkhoev, Dmitriy, et al.
Published: (2025)

Graceful Forgetting in Generative Language Models
by: Jiang, Chunyang, et al.
Published: (2025)

Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoning
by: Liu, Jiashun, et al.
Published: (2025)

Wings: Learning Multimodal LLMs without Text-only Forgetting
by: Zhang, Yi-Kai, et al.
Published: (2024)

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
by: Lai, Song, et al.
Published: (2025)

Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
by: Abdi, Immanuel, et al.
Published: (2026)

LoRA Learns Less and Forgets Less
by: Biderman, Dan, et al.
Published: (2024)

Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting
by: Liu, Chi, et al.
Published: (2026)

SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning
by: Liao, Huanxuan, et al.
Published: (2025)

Pruning Foundation Models for High Accuracy without Retraining
by: Zhao, Pu, et al.
Published: (2024)

FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning
by: Feng, Yujie, et al.
Published: (2026)

Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
by: Choi, Minsik, et al.
Published: (2025)

Offline Learning and Forgetting for Reasoning with Large Language Models
by: Ni, Tianwei, et al.
Published: (2025)

Mapping Post-Training Forgetting in Language Models at Scale
by: Harmon, Jackson, et al.
Published: (2025)