Cover Image

Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ren, Liliang, Liu, Yang, Lu, Yadong, Shen, Yelong, Liang, Chen, Chen, Weizhu
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rethinking Language Model Scaling under Transferable Hypersphere Optimization
by: Ren, Liliang, et al.
Published: (2026)

Shuffle the Context: RoPE-Perturbed Self-Distillation for Long-Context Adaptation
by: Li, Zichong, et al.
Published: (2026)

Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation
by: Ren, Liliang, et al.
Published: (2025)

Reinforcement Learning for Reasoning in Large Language Models with One Training Example
by: Wang, Yiping, et al.
Published: (2025)

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
by: Ma, Xuezhe, et al.
Published: (2024)

OmniParser for Pure Vision Based GUI Agent
by: Lu, Yadong, et al.
Published: (2024)

SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
by: Liang, Xiao, et al.
Published: (2025)

Multi-LoRA Composition for Image Generation
by: Zhong, Ming, et al.
Published: (2024)

Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalities
by: Saporta, Adriel, et al.
Published: (2024)

SambaLingo: Teaching Large Language Models New Languages
by: Csaki, Zoltan, et al.
Published: (2024)

ThetaEvolve: Test-time Learning on Open Problems
by: Wang, Yiping, et al.
Published: (2025)

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment
by: Yin, Yueqin, et al.
Published: (2024)

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
by: Zhan, Zheng, et al.
Published: (2025)

NorMuon: Making Muon more efficient and scalable
by: Li, Zichong, et al.
Published: (2025)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
by: He, Wei, et al.
Published: (2024)

Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
by: Zhong, Ming, et al.
Published: (2023)

Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing
by: Wu, Chen, et al.
Published: (2025)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
by: Li, Zichong, et al.
Published: (2025)

Simple, Efficient and Scalable Structure-aware Adapter Boosts Protein Language Models
by: Tan, Yang, et al.
Published: (2024)

Can Language Models Compose Skills In-Context?
by: Liu, Zidong, et al.
Published: (2025)

Uncertainty Quantification for In-Context Learning of Large Language Models
by: Ling, Chen, et al.
Published: (2024)

Factual Knowledge in Language Models: Robustness and Anomalies under Simple Temporal Context Variations
by: Khodja, Hichem Ammar, et al.
Published: (2025)

FoldGPT: Simple and Effective Large Language Model Compression Scheme
by: Liu, Songwei, et al.
Published: (2024)

Rethinking Token Reduction for State Space Models
by: Zhan, Zheng, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning of State Space Models
by: Galim, Kevin, et al.
Published: (2024)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
by: Yang, Yaming, et al.
Published: (2024)

ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models
by: Liang, Manlai, et al.
Published: (2025)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
by: MiniCPM Team, et al.
Published: (2026)

PICASO: Permutation-Invariant Context Composition with State Space Models
by: Liu, Tian Yu, et al.
Published: (2025)

Core Context Aware Transformers for Long Context Language Modeling
by: Chen, Yaofo, et al.
Published: (2024)

dLLM: Simple Diffusion Language Modeling
by: Zhou, Zhanhui, et al.
Published: (2026)

How to Train Long-Context Language Models (Effectively)
by: Gao, Tianyu, et al.
Published: (2024)

Taipan: Efficient and Expressive State Space Language Models with Selective Attention
by: Van Nguyen, Chien, et al.
Published: (2024)

Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models
by: Nunez, Elvis, et al.
Published: (2024)

Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
by: Shen, Yangyi, et al.
Published: (2026)

Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
by: Wang, Fei, et al.
Published: (2024)

B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory
by: Zancato, Luca, et al.
Published: (2024)

MemMamba: Rethinking Memory Patterns in State Space Model
by: Wang, Youjin, et al.
Published: (2025)

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
by: Qiu, Quantong, et al.
Published: (2026)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
by: Chen, Yukang, et al.
Published: (2023)