Cover Image

Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zheng, Junhao, Qiu, Shengjie, Ma, Qianli
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Can LLMs Learn New Concepts Incrementally without Forgetting?
by: Zheng, Junhao, et al.
Published: (2024)

Towards Lifelong Learning of Large Language Models: A Survey
by: Zheng, Junhao, et al.
Published: (2024)

Incremental Sequence Labeling: A Tale of Two Shifts
by: Qiu, Shengjie, et al.
Published: (2024)

Spurious Forgetting in Continual Learning of Language Models
by: Zheng, Junhao, et al.
Published: (2025)

Self-Adaptive Reconstruction with Contrastive Learning for Unsupervised Sentence Embeddings
by: Liu, Junlong, et al.
Published: (2024)

DEPT: Decoupled Embeddings for Pre-training Language Models
by: Iacob, Alex, et al.
Published: (2024)

Model Merging in Pre-training of Large Language Models
by: Li, Yunshui, et al.
Published: (2025)

Balancing the Causal Effects in Class-Incremental Learning
by: Zheng, Junhao, et al.
Published: (2024)

Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers
by: Kadlčík, Marek, et al.
Published: (2025)

Making Pre-trained Language Models Great on Tabular Prediction
by: Yan, Jiahuan, et al.
Published: (2024)

Mimetic Initialization Helps State Space Models Learn to Recall
by: Trockman, Asher, et al.
Published: (2024)

Structural Pruning of Pre-trained Language Models via Neural Architecture Search
by: Klein, Aaron, et al.
Published: (2024)

Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
by: Wang, Qianli, et al.
Published: (2025)

Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training
by: Chung, Woojin, et al.
Published: (2025)

PaPaformer: Language Model from Pre-trained Parallel Paths
by: Tapaninaho, Joonas, et al.
Published: (2025)

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
by: Thangarasa, Vithursan, et al.
Published: (2024)

Pre-trained Vision-Language Models Learn Discoverable Visual Concepts
by: Zang, Yuan, et al.
Published: (2024)

Cross-Refine: Improving Natural Language Explanation Generation by Learning in Tandem
by: Wang, Qianli, et al.
Published: (2024)

Fine-Tuning Pre-trained Language Models to Detect In-Game Trash Talks
by: Fesalbon, Daniel, et al.
Published: (2024)

Pre-trained Large Language Models Use Fourier Features to Compute Addition
by: Zhou, Tianyi, et al.
Published: (2024)

Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
by: Sharma, Kartik, et al.
Published: (2025)

HLAT: High-quality Large Language Model Pre-trained on AWS Trainium
by: Fan, Haozheng, et al.
Published: (2024)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
by: Zhang, Ying, et al.
Published: (2024)

Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training
by: Zhong, Zexuan, et al.
Published: (2024)

Sequence-to-Sequence Spanish Pre-trained Language Models
by: Araujo, Vladimir, et al.
Published: (2023)

Investigating Data Contamination for Pre-training Language Models
by: Jiang, Minhao, et al.
Published: (2024)

Aligning Pre-trained Models for Spoken Language Translation
by: Sedláček, Šimon, et al.
Published: (2024)

Towards Incremental Learning in Large Language Models: A Critical Review
by: Jovanovic, Mladjan, et al.
Published: (2024)

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
by: Liu, Hong, et al.
Published: (2023)

Evaluating the Effectiveness of Pre-trained Language Models in Predicting the Helpfulness of Online Product Reviews
by: Boluki, Ali, et al.
Published: (2023)

A Context-Aware Approach for Enhancing Data Imputation with Pre-trained Language Models
by: Hayat, Ahatsham, et al.
Published: (2024)

Verbalized Machine Learning: Revisiting Machine Learning with Language Models
by: Xiao, Tim Z., et al.
Published: (2024)

Efficient Continual Pre-training of LLMs for Low-resource Languages
by: Nag, Arijit, et al.
Published: (2024)

The Dark Side of the Language: Pre-trained Transformers in the DarkNet
by: Ranaldi, Leonardo, et al.
Published: (2022)

Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning
by: Yano, Kazuki, et al.
Published: (2026)

Revisiting In-Context Learning with Long Context Language Models
by: Baek, Jinheon, et al.
Published: (2024)

Integrating Pre-trained Language Model into Neural Machine Translation
by: Hwang, Soon-Jae, et al.
Published: (2023)

CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
by: Gu, Jiawei, et al.
Published: (2024)

Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
by: Chen, Yuyan, et al.
Published: (2024)

Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models
by: Lou, Xingzhou, et al.
Published: (2024)