Copertina

Examining Forgetting in Continual Pre-training of Aligned Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Chen-An, Lee, Hung-Yi
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Exploring Forgetting in Large Language Model Pre-Training
di: Liao, Chonghua, et al.
Pubblicazione: (2024)

Efficient Continual Pre-training for Building Domain Specific Large Language Models
di: Xie, Yong, et al.
Pubblicazione: (2023)

Aligning Pre-trained Models for Spoken Language Translation
di: Sedláček, Šimon, et al.
Pubblicazione: (2024)

Model Merging in Pre-training of Large Language Models
di: Li, Yunshui, et al.
Pubblicazione: (2025)

Towards Effective and Efficient Continual Pre-training of Large Language Models
di: Chen, Jie, et al.
Pubblicazione: (2024)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
di: Ibrahim, Adam, et al.
Pubblicazione: (2024)

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
di: Luo, Yun, et al.
Pubblicazione: (2023)

Construction of Domain-specified Japanese Large Language Model for Finance through Continual Pre-training
di: Hirano, Masanori, et al.
Pubblicazione: (2024)

Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
di: Lu, Ke-Han, et al.
Pubblicazione: (2025)

From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan
di: Yang, Lei, et al.
Pubblicazione: (2025)

OntoTune: Ontology-Driven Self-training for Aligning Large Language Models
di: Liu, Zhiqiang, et al.
Pubblicazione: (2025)

Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
di: Tu, Zhijun, et al.
Pubblicazione: (2025)

Making Pre-trained Language Models Better Continual Few-Shot Relation Extractors
di: Ma, Shengkun, et al.
Pubblicazione: (2024)

TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
di: Tseng, Liang-Hsuan, et al.
Pubblicazione: (2025)

Cross-layer Attention Sharing for Pre-trained Large Language Models
di: Mu, Yongyu, et al.
Pubblicazione: (2024)

Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
di: Ma, Shengjie, et al.
Pubblicazione: (2025)

ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data
di: Chen, Tong, et al.
Pubblicazione: (2025)

MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science
di: Kim, Junho, et al.
Pubblicazione: (2024)

SongSage: A Large Musical Language Model with Lyric Generative Pre-training
di: Guo, Jiani, et al.
Pubblicazione: (2026)

Scaling Agents via Continual Pre-training
di: Su, Liangcai, et al.
Pubblicazione: (2025)

STEP: Staged Parameter-Efficient Pre-training for Large Language Models
di: Yano, Kazuki, et al.
Pubblicazione: (2025)

The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models
di: Liu, Yan, et al.
Pubblicazione: (2024)

FIT to Forget: Robust Continual Unlearning for Large Language Models
di: Xu, Xiaoyu, et al.
Pubblicazione: (2026)

Metadata Conditioning Accelerates Language Model Pre-training
di: Gao, Tianyu, et al.
Pubblicazione: (2025)

SparseLLM: Towards Global Pruning for Pre-trained Language Models
di: Bai, Guangji, et al.
Pubblicazione: (2024)

Pre-trained Large Language Models for Financial Sentiment Analysis
di: Luo, Wei, et al.
Pubblicazione: (2024)

Spike No More: Stabilizing the Pre-training of Large Language Models
di: Takase, Sho, et al.
Pubblicazione: (2023)

Continual Forgetting for Pre-trained Vision Models
di: Zhao, Hongbo, et al.
Pubblicazione: (2024)

Revisiting Catastrophic Forgetting in Large Language Model Tuning
di: Li, Hongyu, et al.
Pubblicazione: (2024)

Non-instructional Fine-tuning: Enabling Instruction-Following Capabilities in Pre-trained Language Models without Instruction-Following Data
di: Xie, Juncheng, et al.
Pubblicazione: (2024)

TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
di: Tseng, Liang-Hsuan, et al.
Pubblicazione: (2026)

MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector
di: Fu, Wenjie, et al.
Pubblicazione: (2024)

IterAlign: Iterative Constitutional Alignment of Large Language Models
di: Chen, Xiusi, et al.
Pubblicazione: (2024)

Can Pre-trained Language Models Understand Chinese Humor?
di: Chen, Yuyan, et al.
Pubblicazione: (2024)

Talking to Yourself: Defying Forgetting in Large Language Models
di: Sun, Yutao, et al.
Pubblicazione: (2026)

Pre-trained Language Models Return Distinguishable Probability Distributions to Unfaithfully Hallucinated Texts
di: Cha, Taehun, et al.
Pubblicazione: (2024)

Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models
di: Ni, Shiwen, et al.
Pubblicazione: (2023)

Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models
di: He, Yu, et al.
Pubblicazione: (2025)

Vi-Mistral-X: Building a Vietnamese Language Model with Advanced Continual Pre-training
di: Vo, James
Pubblicazione: (2024)

Pre-training Distillation for Large Language Models: A Design Space Exploration
di: Peng, Hao, et al.
Pubblicazione: (2024)