Cover Image

Improving Language Plasticity via Pretraining with Active Forgetting

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Yihong, Marchisio, Kelly, Raileanu, Roberta, Adelani, David Ifeoluwa, Stenetorp, Pontus, Riedel, Sebastian, Artetxe, Mikel
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Machine Learning Neural and Evolutionary Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Language Models Learn Universal Representations of Numbers and Here's Why You Should Care
by: Štefánik, Michal, et al.
Published: (2025)

Personalized Language Models via Privacy-Preserving Evolutionary Model Merging
by: Kim, Kyuyoung, et al.
Published: (2025)

Simulated Language Acquisition in a Biologically Realistic Model of the Brain
by: Mitropolsky, Daniel, et al.
Published: (2025)

Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Brain
by: An, Jingmin, et al.
Published: (2025)

Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks
by: Shen, Guobin, et al.
Published: (2023)

SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking
by: Xing, Xingrun, et al.
Published: (2024)

Agent Skill Acquisition for Large Language Models via CycleQD
by: Kuroki, So, et al.
Published: (2024)

LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimization
by: Nasir, Muhammad U., et al.
Published: (2023)

Interpreting and learning voice commands with a Large Language Model for a robot system
by: Stankevich, Stanislau, et al.
Published: (2024)

Projection-Free Evolution Strategies for Continuous Prompt Search
by: Cai, Yu, et al.
Published: (2026)

What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search
by: Zhang, Xinhao, et al.
Published: (2026)

A Comprehensive Empirical Evaluation of Existing Word Embedding Approaches
by: Zaland, Obaidullah, et al.
Published: (2023)

Generalization and Completeness of Stochastic Local Search Algorithms
by: Loscos, Daniel, et al.
Published: (2026)

Spiking Convolutional Neural Networks for Text Classification
by: Lv, Changze, et al.
Published: (2024)

A survey on learning models of spiking neural membrane systems and spiking neural networks
by: Paul, Prithwineel, et al.
Published: (2024)

Children's Acquisition of Tail-recursion Sequences: A Review of Locative Recursion and Possessive Recursion as Examples
by: Wang, Xiaoyi, et al.
Published: (2024)

Towards Faster k-Nearest-Neighbor Machine Translation
by: Shi, Xiangyu, et al.
Published: (2023)

Bio-Inspired Mamba: Temporal Locality and Bioplausible Learning in Selective State Space Models
by: Qin, Jiahao
Published: (2024)

VietNormalizer: An Open-Source, Dependency-Free Python Library for Vietnamese Text Normalization in TTS and NLP Applications
by: Nguyen, Hung Vu, et al.
Published: (2026)

SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
by: Xing, Xingrun, et al.
Published: (2024)

DARWIN: Dynamic Agentically Rewriting Self-Improving Network
by: Jiang, Henry
Published: (2026)

Synchrony-Gated Plasticity with Dopamine Modulation for Spiking Neural Networks
by: Tian, Yuchen, et al.
Published: (2025)

SpikeCLIP: A Contrastive Language-Image Pretrained Spiking Neural Network
by: Lv, Changze, et al.
Published: (2023)

Improving Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches
by: Saxena, Aditya, et al.
Published: (2024)

SNAP: Stopping Catastrophic Forgetting in Hebbian Learning with Sigmoidal Neuronal Adaptive Plasticity
by: Xu, Tianyi, et al.
Published: (2024)

MorphSNN: Adaptive Graph Diffusion and Structural Plasticity for Spiking Neural Networks
by: Huang, Yongsheng, et al.
Published: (2026)

Large Language Models for Tuning Evolution Strategies
by: Kramer, Oliver
Published: (2024)

ETLP: Event-based Three-factor Local Plasticity for online learning with neuromorphic hardware
by: Quintana, Fernando M., et al.
Published: (2023)

Continual Learning with Hebbian Plasticity in Sparse and Predictive Coding Networks: A Survey and Perspective
by: Safa, Ali
Published: (2024)

Towards Explainable Evolution Strategies with Large Language Models
by: Baumann, Jill, et al.
Published: (2024)

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
by: Wang, Hanrui, et al.
Published: (2020)

Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap
by: Wu, Xingyu, et al.
Published: (2024)

UniBCI: Towards a Unified Pretrained Model for Invasive Brain-Computer Interfaces
by: Hong, Binjie, et al.
Published: (2026)

Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers
by: Kadlčík, Marek, et al.
Published: (2025)

ToxSearch: Evolving Prompts for Toxicity Search in Large Language Models
by: Shelar, Onkar, et al.
Published: (2025)

PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding
by: Li, Kangcong, et al.
Published: (2025)

Sorbet: A Neuromorphic Hardware-Compatible Transformer-Based Spiking Language Model
by: Tang, Kaiwen, et al.
Published: (2024)

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks
by: Zhu, Rui-Jie, et al.
Published: (2023)

Improving genetic algorithms performance via deterministic population shrinkage
by: Laredo, Juan Luis Jiménez, et al.
Published: (2024)

Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models
by: Dong, Peijie, et al.
Published: (2024)