Empirical Study on Updating Key-Value Memories in Transformer Feed-forward Layers

Documenti analoghi

• A Closer Look into Mixture-of-Experts in Large Language Models
  di: Lo, Ka Man, et al.
  Pubblicazione: (2024)
• Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training
  di: Du, Wenyu, et al.
  Pubblicazione: (2024)
• Layerwise Recurrent Router for Mixture-of-Experts
  di: Qiu, Zihan, et al.
  Pubblicazione: (2024)
• Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers
  di: Chen, Lei, et al.
  Pubblicazione: (2024)
• F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translation
  di: Wu, Junhong, et al.
  Pubblicazione: (2024)