Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting

Documenti analoghi

• Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
  di: Abdi, Immanuel, et al.
  Pubblicazione: (2026)
• Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
  di: Ren, Weijieying, et al.
  Pubblicazione: (2024)
• Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
  di: Kenneweg, Philip, et al.
  Pubblicazione: (2024)
• Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
  di: Li, Junzhuo, et al.
  Pubblicazione: (2025)
• CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
  di: Fawi, Muhammad
  Pubblicazione: (2024)