Image de couverture de livre

On Fine-Grained I/O Complexity of Attention Backward Passes

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Xiaoyu, Liang, Yingyu, Shi, Zhenmei, Song, Zhao, Yue, Song, Zhang, Jiahao
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence Computational Complexity Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Circuit Complexity Bounds for Visual Autoregressive Model
par: Ke, Yekun, et autres
Publié: (2025)

Theoretical Constraints on the Expressive Power of $\mathsf{RoPE}$-based Tensor Attention Transformers
par: Li, Xiaoyu, et autres
Publié: (2024)

The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity
par: Chen, Yifang, et autres
Publié: (2024)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
par: Chen, Bo, et autres
Publié: (2024)

RoPE Attention Can Be Trained in Almost Linear Time
par: Cao, Yang, et autres
Publié: (2024)

On the Computational Capability of Graph Neural Networks: A Circuit Complexity Bound Perspective
par: Li, Xiaoyu, et autres
Publié: (2025)

Modern Hopfield Networks Require Chain-of-Thought to Solve $\mathsf{NC}^1$-Hard Problems
par: Cao, Yang, et autres
Publié: (2024)

Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decoding
par: Chen, Yifang, et autres
Publié: (2025)

On Computational Limits and Provably Efficient Criteria of Visual Autoregressive Models: A Fine-Grained Complexity Analysis
par: Ke, Yekun, et autres
Publié: (2025)

Neural Algorithmic Reasoning for Hypergraphs with Looped Transformers
par: Huang, Zekai, et autres
Publié: (2025)

When Can We Solve the Weighted Low Rank Approximation Problem in Truly Subquadratic Time?
par: Li, Chenyang, et autres
Publié: (2025)

Looped ReLU MLPs May Be All You Need as Practical Programmable Computers
par: Liang, Yingyu, et autres
Publié: (2024)

Provable Failure of Language Models in Learning Majority Boolean Logic via Gradient Descent
par: Chen, Bo, et autres
Publié: (2025)

A Tighter Complexity Analysis of SparseGPT
par: Li, Xiaoyu, et autres
Publié: (2024)

Training Tensor Attention Efficiently: From Cubic to Almost Linear Time
par: Cao, Yang, et autres
Publié: (2024)

HSR-Enhanced Sparse Attention Acceleration
par: Chen, Bo, et autres
Publié: (2024)

Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix
par: Liang, Yingyu, et autres
Publié: (2024)

On Computational Limits of FlowAR Models: Expressivity and Efficiency
par: Cao, Yang, et autres
Publié: (2025)

Towards Infinite-Long Prefix in Transformer
par: Liang, Yingyu, et autres
Publié: (2024)

Universal Approximation of Visual Autoregressive Transformers
par: Chen, Yifang, et autres
Publié: (2025)

Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Time
par: Liang, Yingyu, et autres
Publié: (2024)

Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability
par: Xu, Zhuoyan, et autres
Publié: (2024)

Unlocking the Theory Behind Scaling 1-Bit Neural Networks
par: Daliri, Majid, et autres
Publié: (2024)

The Fine-Grained Complexity of Gradient Computation for Training Large Language Models
par: Alman, Josh, et autres
Publié: (2024)

Why Larger Language Models Do In-context Learning Differently?
par: Shi, Zhenmei, et autres
Publié: (2024)

RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation
par: Cao, Yuefan, et autres
Publié: (2025)

NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes
par: Fan, Lizhou, et autres
Publié: (2023)

Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction
par: Shi, Zhenmei, et autres
Publié: (2024)

Dissecting Submission Limit in Desk-Rejections: A Mathematical Analysis of Fairness in AI Conference Policies
par: Cao, Yuefan, et autres
Publié: (2025)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Provable Differentially Private Computation of the Cross-Attention Mechanism
par: Ke, Yekun, et autres
Publié: (2024)

Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective
par: Liang, Yingyu, et autres
Publié: (2024)

Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning
par: Xu, Zhuoyan, et autres
Publié: (2024)

A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits
par: Zhang, Yuyang, et autres
Publié: (2026)

A Provable Expressiveness Hierarchy in Hybrid Linear-Full Attention
par: Ye, Xiaowei, et autres
Publié: (2026)

T2VTextBench: A Human Evaluation Benchmark for Textual Control in Video Generation Models
par: Guo, Xuyang, et autres
Publié: (2025)

T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generation
par: Guo, Xuyang, et autres
Publié: (2025)

Visual Autoregressive Transformers Must Use $Ω(n^2 d)$ Memory
par: Cao, Yang, et autres
Publié: (2025)

Demystifying the unreasonable effectiveness of online alignment methods
par: Kang, Enoch Hyunwook
Publié: (2026)

Text-to-Image Diffusion Models Cannot Count, and Prompt Refinement Cannot Help
par: Guo, Xuyang, et autres
Publié: (2025)