Copertina

Understanding and Improving Length Generalization in Recurrent Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Ruiz, Ricardo Buitrago, Gu, Albert
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

On the Benefits of Memory for Modeling Time-Dependent PDEs
di: Ruiz, Ricardo Buitrago, et al.
Pubblicazione: (2024)

Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism
di: Bick, Aviv, et al.
Pubblicazione: (2025)

Length Generalization with Log-Depth Recurrent Units
di: Pert, Charles, et al.
Pubblicazione: (2026)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
di: Leng, Jiaqi, et al.
Pubblicazione: (2025)

Improving Variable-Length Generation in Diffusion Language Models via Length Regularization
di: Cheng, Zicong, et al.
Pubblicazione: (2026)

Llamba: Scaling Distilled Recurrent Models for Efficient Language Processing
di: Bick, Aviv, et al.
Pubblicazione: (2025)

A Formal Framework for Understanding Length Generalization in Transformers
di: Huang, Xinting, et al.
Pubblicazione: (2024)

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
di: Dao, Tri, et al.
Pubblicazione: (2024)

Delayed Attention Training Improves Length Generalization in Transformer--RNN Hybrids
di: Phan, Buu, et al.
Pubblicazione: (2025)

Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes
di: Jeon, Dongjae, et al.
Pubblicazione: (2024)

Channel-Wise MLPs Improve the Generalization of Recurrent Convolutional Networks
di: Breslow, Nathan
Pubblicazione: (2025)

Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
di: Lee, Nayoung, et al.
Pubblicazione: (2025)

Predicting the Stay Length of Patients in Hospitals using Convolutional Gated Recurrent Deep Learning Model
di: Neshat, Mehdi, et al.
Pubblicazione: (2024)

Non-Asymptotic Length Generalization
di: Chen, Thomas, et al.
Pubblicazione: (2025)

Looped Transformers for Length Generalization
di: Fan, Ying, et al.
Pubblicazione: (2024)

Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers
di: Hwang, Sukjun, et al.
Pubblicazione: (2024)

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling
di: Hwang, Sukjun, et al.
Pubblicazione: (2025)

Mamba: Linear-Time Sequence Modeling with Selective State Spaces
di: Gu, Albert, et al.
Pubblicazione: (2023)

Quantitative Bounds for Length Generalization in Transformers
di: Izzo, Zachary, et al.
Pubblicazione: (2025)

Universal Length Generalization with Turing Programs
di: Hou, Kaiying, et al.
Pubblicazione: (2024)

Mamba-3: Improved Sequence Modeling using State Space Principles
di: Lahoti, Aakash, et al.
Pubblicazione: (2026)

Adaptive Length Image Tokenization via Recurrent Allocation
di: Duggal, Shivam, et al.
Pubblicazione: (2024)

To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models
di: Malach, Eran, et al.
Pubblicazione: (2025)

ARC-AGI Without Pretraining
di: Liao, Isaac, et al.
Pubblicazione: (2025)

Interpreting Affine Recurrence Learning in GPT-style Transformers
di: Bhargav, Samarth, et al.
Pubblicazione: (2024)

Learning Variable-Length Tokenization for Generative Recommendation
di: Wang, Minhao, et al.
Pubblicazione: (2026)

Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
di: Cho, Hanseul, et al.
Pubblicazione: (2024)

On Provable Length and Compositional Generalization
di: Ahuja, Kartik, et al.
Pubblicazione: (2024)

Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
di: Cho, Hanseul, et al.
Pubblicazione: (2024)

Fragility-aware Classification for Understanding Risk and Improving Generalization
di: Yang, Chen, et al.
Pubblicazione: (2025)

Low-Dimension-to-High-Dimension Generalization And Its Implications for Length Generalization
di: Chen, Yang, et al.
Pubblicazione: (2024)

Generalization and Risk Bounds for Recurrent Neural Networks
di: Cheng, Xuewei, et al.
Pubblicazione: (2024)

Probing Length Generalization in Mamba via Image Reconstruction
di: Rathjens, Jan, et al.
Pubblicazione: (2026)

Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks
di: Sieber, Jerome, et al.
Pubblicazione: (2024)

A Theory of Machine Understanding via the Minimum Description Length Principle
di: Zhang, Canlin, et al.
Pubblicazione: (2025)

Uncertainty-Gated Generative Modeling
di: Gu, Xingrui, et al.
Pubblicazione: (2026)

Understanding Dynamic Compute Allocation in Recurrent Transformers
di: Moosa, Ibraheem Muhammad, et al.
Pubblicazione: (2026)

Bi-directional Recurrence Improves Transformer in Partially Observable Markov Decision Processes
di: Arora, Ashok, et al.
Pubblicazione: (2025)

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
di: De, Soham, et al.
Pubblicazione: (2024)

On Vanishing Variance in Transformer Length Generalization
di: Li, Ruining, et al.
Pubblicazione: (2025)