Cover Image

Exploring Forgetting in Large Language Model Pre-Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liao, Chonghua, Xie, Ruobing, Sun, Xingwu, Sun, Haowen, Kang, Zhanhui
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Training
by: Ma, Haokai, et al.
Published: (2025)

Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization
by: Fu, Yuhan, et al.
Published: (2024)

Language Models "Grok" to Copy
by: Lv, Ang, et al.
Published: (2024)

The Climb Carves Wisdom Deeper Than the Summit: On the Noisy Rewards in Learning to Reason
by: Lv, Ang, et al.
Published: (2025)

Continuous Speech Tokenizer in Text To Speech
by: Li, Yixing, et al.
Published: (2024)

Lossless KV Cache Compression to 2%
by: Yang, Zhen, et al.
Published: (2024)

QAVA: Query-Agnostic Visual Attack to Large Vision-Language Models
by: Zhang, Yudong, et al.
Published: (2025)

The Elephant in the Room: Rethinking the Usage of Pre-trained Language Model in Sequential Recommendation
by: Qu, Zekai, et al.
Published: (2024)

More Expressive Attention with Negative Weights
by: Lv, Ang, et al.
Published: (2024)

Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions
by: Fu, Yuhan, et al.
Published: (2024)

Autonomy-of-Experts Models
by: Lv, Ang, et al.
Published: (2025)

Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
by: Chen, Zhongzhi, et al.
Published: (2023)

HMoE: Heterogeneous Mixture of Experts for Language Modeling
by: Wang, An, et al.
Published: (2024)

DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language Models
by: Zhang, Yudong, et al.
Published: (2024)

Self-Distillation for Multi-Token Prediction
by: Zhao, Guoliang, et al.
Published: (2026)

Enhancing Contrastive Learning Inspired by the Philosophy of "The Blind Men and the Elephant"
by: Zhang, Yudong, et al.
Published: (2024)

The Security Threat of Compressed Projectors in Large Vision-Language Models
by: Zhang, Yudong, et al.
Published: (2025)

Diverse and Fine-Grained Instruction-Following Ability Exploration with Synthetic Data
by: Gu, Zihui, et al.
Published: (2024)

Multi-Grained Patch Training for Efficient LLM-based Recommendation
by: Liao, Jiayi, et al.
Published: (2025)

Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models
by: Wang, Huazheng, et al.
Published: (2025)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
by: Li, Chen-An, et al.
Published: (2024)

Scaling Laws for Floating Point Quantization Training
by: Sun, Xingwu, et al.
Published: (2025)

Fighting Fire with Fire (F3): A Training-free and Efficient Visual Adversarial Example Purification Method in LVLMs
by: Zhang, Yudong, et al.
Published: (2025)

Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
by: Chang, Tyler A., et al.
Published: (2023)

Proximal Supervised Fine-Tuning
by: Zhu, Wenhong, et al.
Published: (2025)

Hybrid-Tower: Fine-grained Pseudo-query Interaction and Generation for Text-to-Video Retrieval
by: Lan, Bangxiang, et al.
Published: (2025)

Exploring the Benefit of Activation Sparsity in Pre-training
by: Zhang, Zhengyan, et al.
Published: (2024)

Amuro and Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models
by: Sun, Kaiser, et al.
Published: (2024)

Talking to Yourself: Defying Forgetting in Large Language Models
by: Sun, Yutao, et al.
Published: (2026)

Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models
by: Zhu, Didi, et al.
Published: (2024)

Unveiling and Addressing Pseudo Forgetting in Large Language Models
by: Sun, Huashan, et al.
Published: (2024)

PIP: Detecting Adversarial Examples in Large Vision-Language Models via Attention Patterns of Irrelevant Probe Questions
by: Zhang, Yudong, et al.
Published: (2024)

MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting
by: Li, Tianhao, et al.
Published: (2024)

PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset
by: Liu, Jiazhen, et al.
Published: (2024)

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal
by: Huang, Jianheng, et al.
Published: (2024)

Exploring the Nexus of Large Language Models and Legal Systems: A Short Survey
by: Qin, Weicong, et al.
Published: (2024)

Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language Models
by: Sawada, Tomohiro, et al.
Published: (2025)

Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers
by: Pan, Haowen, et al.
Published: (2023)

Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models
by: Zhang, Jingyang, et al.
Published: (2024)

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
by: Xiao, Chaojun, et al.
Published: (2023)