Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls

Documenti analoghi

• From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
  di: Deng, Yuntian, et al.
  Pubblicazione: (2024)
• A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity
  di: Lee, Andrew, et al.
  Pubblicazione: (2024)
• Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions
  di: Lee, Andrew, et al.
  Pubblicazione: (2026)
• Relational Composition in Neural Networks: A Survey and Call to Action
  di: Wattenberg, Martin, et al.
  Pubblicazione: (2024)
• What Does it Mean for a Neural Network to Learn a "World Model"?
  di: Li, Kenneth, et al.
  Pubblicazione: (2025)