Copertina

Approximation Rate of the Transformer Architecture for Sequence Modeling

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Jiang, Haotian, Li, Qianxiao
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

The Effect of Attention Head Count on Transformer Approximation
di: Yu, Penghao, et al.
Pubblicazione: (2025)

Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functions
di: Jiang, Haotian, et al.
Pubblicazione: (2025)

InfoFlow: A Framework for Multi-Layer Transformer Analysis
di: Yu, Penghao, et al.
Pubblicazione: (2026)

The Effect of Depth on the Expressivity of Deep Linear State-Space Models
di: Bao, Zeyu, et al.
Pubblicazione: (2025)

Allocation of Parameters in Transformers
di: Yu, Ruoxi, et al.
Pubblicazione: (2025)

From Generalization Analysis to Optimization Designs for State Space Models
di: Liu, Fusheng, et al.
Pubblicazione: (2024)

Autocorrelation Matters: Understanding the Role of Initialization Schemes for State Space Models
di: Liu, Fusheng, et al.
Pubblicazione: (2024)

Inverse Approximation Theory for Nonlinear Recurrent Neural Networks
di: Wang, Shida, et al.
Pubblicazione: (2023)

Learning task-specific predictive models for scientific computing
di: Yin, Jianyuan, et al.
Pubblicazione: (2025)

Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving
di: Arisaka, Sohei, et al.
Pubblicazione: (2024)

Unifying back-propagation and forward-forward algorithms through model predictive control
di: Ren, Lianhai, et al.
Pubblicazione: (2024)

StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization
di: Wang, Shida, et al.
Pubblicazione: (2023)

DynGMA: a robust approach for learning stochastic differential equations from data
di: Zhu, Aiqing, et al.
Pubblicazione: (2024)

Learning Macroscopic Dynamics from Partial Microscopic Observations
di: Chen, Mengyi, et al.
Pubblicazione: (2024)

Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Input
di: Takakura, Shokichi, et al.
Pubblicazione: (2023)

Transportation Marketplace Rate Forecast Using Signature Transform
di: Gu, Haotian, et al.
Pubblicazione: (2024)

Continuity-Preserving Convolutional Autoencoders for Learning Continuous Latent Dynamical Models from Images
di: Zhu, Aiqing, et al.
Pubblicazione: (2025)

Learning Permutation-invariant Macroscopic Dynamics
di: Han, Zhichao, et al.
Pubblicazione: (2026)

On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis
di: Li, Zhong, et al.
Pubblicazione: (2020)

Machine Unlearning under Retain-Forget Entanglement
di: Cheng, Jingpu, et al.
Pubblicazione: (2026)

Architectural and Inferential Inductive Biases For Exchangeable Sequence Modeling
di: Mittal, Daksh, et al.
Pubblicazione: (2025)

Introduction to Sequence Modeling with Transformers
di: Kämäräinen, Joni-Kristian
Pubblicazione: (2025)

MSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank Restoration
di: Ren, Lianhai, et al.
Pubblicazione: (2026)

TransMamba: A Sequence-Level Hybrid Transformer-Mamba Language Model
di: Li, Yixing, et al.
Pubblicazione: (2025)

Scalable learning of macroscopic stochastic dynamics
di: Chen, Mengyi, et al.
Pubblicazione: (2025)

A unified framework for establishing the universal approximation of transformer-type architectures
di: Cheng, Jingpu, et al.
Pubblicazione: (2025)

Deep learning and the rate of approximation by flows
di: Cheng, Jingpu, et al.
Pubblicazione: (2026)

An All-MLP Sequence Modeling Architecture That Excels at Copying
di: Cui, Chenwei, et al.
Pubblicazione: (2024)

Local Attention Mechanism: Boosting the Transformer Architecture for Long-Sequence Time Series Forecasting
di: Aguilera-Martos, Ignacio, et al.
Pubblicazione: (2024)

Adaptive SGD with Line-Search and Polyak Stepsizes: Nonconvex Convergence and Accelerated Rates
di: Wu, Haotian
Pubblicazione: (2025)

Identifiable learning of dissipative dynamics
di: Zhu, Aiqing, et al.
Pubblicazione: (2025)

Mitigating distribution shift in machine learning-augmented hybrid simulation
di: Zhao, Jiaxi, et al.
Pubblicazione: (2024)

ELO-Rated Sequence Rewards: Advancing Reinforcement Learning Models
di: Ju, Qi, et al.
Pubblicazione: (2024)

Sequence-to-Sequence Models with Attention Mechanistically Map to the Architecture of Human Memory Search
di: Salvatore, Nikolaus, et al.
Pubblicazione: (2025)

Modeling Long Sequences in Bladder Cancer Recurrence: A Comparative Evaluation of LSTM,Transformer,and Mamba
di: Zhang, Runquan, et al.
Pubblicazione: (2024)

Sequence Complementor: Complementing Transformers For Time Series Forecasting with Learnable Sequences
di: Chen, Xiwen, et al.
Pubblicazione: (2025)

Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling
di: Wang, Mingze, et al.
Pubblicazione: (2024)

Lyra: An Efficient and Expressive Subquadratic Architecture for Modeling Biological Sequences
di: Ramesh, Krithik, et al.
Pubblicazione: (2025)

Terminally constrained flow-based generative models from an optimal control perspective
di: Gao, Weiguo, et al.
Pubblicazione: (2026)

HAD: Hybrid Architecture Distillation Outperforms Teacher in Genomic Sequence Modeling
di: Yang, Hexiong, et al.
Pubblicazione: (2025)