Theoretical Constraints on the Expressive Power of $\mathsf{RoPE}$-based Tensor Attention Transformers

Documenti analoghi

• Circuit Complexity Bounds for RoPE-based Transformer Architecture
  di: Chen, Bo, et al.
  Pubblicazione: (2024)
• RoPE Attention Can Be Trained in Almost Linear Time
  di: Cao, Yang, et al.
  Pubblicazione: (2024)
• Modern Hopfield Networks Require Chain-of-Thought to Solve $\mathsf{NC}^1$-Hard Problems
  di: Cao, Yang, et al.
  Pubblicazione: (2024)
• Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decoding
  di: Chen, Yifang, et al.
  Pubblicazione: (2025)
• On Fine-Grained I/O Complexity of Attention Backward Passes
  di: Li, Xiaoyu, et al.
  Pubblicazione: (2024)