Cover Image

Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nunez, Elvis, Zancato, Luca, Bowman, Benjamin, Golatkar, Aditya, Xia, Wei, Soatto, Stefano
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory
by: Zancato, Luca, et al.
Published: (2024)

Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression
by: Peng, Liangzu, et al.
Published: (2025)

PICASO: Permutation-Invariant Context Composition with State Space Models
by: Liu, Tian Yu, et al.
Published: (2025)

Learning When to Attend: Conditional Memory Access for Long-Context LLMs
by: Choudhary, Sakshi, et al.
Published: (2026)

Priming: Hybrid State Space Models From Pre-trained Transformers
by: Chattopadhyay, Aditya, et al.
Published: (2026)

Maximally-Informative Retrieval for State Space Model Generation
by: Becker, Evan, et al.
Published: (2025)

Linear Spaces of Meanings: Compositional Structures in Vision-Language Models
by: Trager, Matthew, et al.
Published: (2023)

Tangent Transformers for Composition, Privacy and Removal
by: Liu, Tian Yu, et al.
Published: (2023)

Training Data Protection with Compositional Diffusion Models
by: Golatkar, Aditya, et al.
Published: (2023)

Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning
by: Zabounidis, Renos, et al.
Published: (2025)

Learning to Focus: Focal Attention for Selective and Scalable Transformers
by: Ram, Dhananjay, et al.
Published: (2025)

CPR: Retrieval Augmented Generation for Copyright Protection
by: Golatkar, Aditya, et al.
Published: (2024)

Compositional Structures in Neural Embedding and Interaction Decompositions
by: Trager, Matthew, et al.
Published: (2024)

Multi-Modal Hallucination Control by Visual Information Grounding
by: Favero, Alessandro, et al.
Published: (2024)

Cycles of Thought: Measuring LLM Confidence through Stable Explanations
by: Becker, Evan, et al.
Published: (2024)

STree: Speculative Tree Decoding for Hybrid State-Space Models
by: Wu, Yangchao, et al.
Published: (2025)

Experience-Guided Adaptation of Inference-Time Reasoning Strategies
by: Stein, Adam, et al.
Published: (2025)

Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models
by: Liu, Xiaoze, et al.
Published: (2026)

Meanings and Feelings of Large Language Models: Observability of Latent States in Generative AI
by: Liu, Tian Yu, et al.
Published: (2024)

ExecTune: Effective Steering of Black-Box LLMs with Guide Models
by: Lingam, Vijay, et al.
Published: (2026)

Diffusion Soup: Model Merging for Text-to-Image Diffusion Models
by: Biggs, Benjamin, et al.
Published: (2024)

Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
by: Ren, Liliang, et al.
Published: (2024)

Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
by: Pham, Bao, et al.
Published: (2026)

An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models
by: Mundra, Nandini, et al.
Published: (2024)

MemMamba: Rethinking Memory Patterns in State Space Model
by: Wang, Youjin, et al.
Published: (2025)

Fewer Truncations Improve Language Modeling
by: Ding, Hantian, et al.
Published: (2024)

Unveiling the Mystery of Weight in Large Foundation Models: Gaussian Distribution Never Fades
by: Si, Chongjie, et al.
Published: (2025)

Language Model Memory and Memory Models for Language
by: Badger, Benjamin L.
Published: (2026)

MatMamba: A Matryoshka State Space Model
by: Shukla, Abhinav, et al.
Published: (2024)

Hidden States Know Where Reasoning Diverges: Credit Assignment via Span-Level Wasserstein Distance
by: Chen, Xinzhu, et al.
Published: (2026)

MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Models
by: Xia, Kejing, et al.
Published: (2026)

Scaling Linear Attention with Sparse State Expansion
by: Pan, Yuqi, et al.
Published: (2025)

Scaling Bidirectional Spans and Span Violations in Attention Mechanism
by: Kim, Jongwook, et al.
Published: (2025)

FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions
by: Weller, Orion, et al.
Published: (2024)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
by: He, Wei, et al.
Published: (2024)

Rethinking Token Reduction for State Space Models
by: Zhan, Zheng, et al.
Published: (2024)

Masked Mixers for Language Generation and Retrieval
by: Badger, Benjamin L.
Published: (2024)

Parameter-Efficient Fine-Tuning of State Space Models
by: Galim, Kevin, et al.
Published: (2024)

Momentum Posterior Regularization for Multi-hop Dense Retrieval
by: Xia, Zehua, et al.
Published: (2024)

AI Agents as Universal Task Solvers
by: Achille, Alessandro, et al.
Published: (2025)