Cover Image

LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ye, Zhifan, Xia, Kejing, Fu, Yonggan, Dong, Xin, Hong, Jihoon, Yuan, Xiangchi, Diao, Shizhe, Kautz, Jan, Molchanov, Pavlo, Lin, Yingyan Celine
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
by: Shi, Dachuan, et al.
Published: (2025)

Small Language Models are the Future of Agentic AI
by: Belcak, Peter, et al.
Published: (2025)

Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
by: Fu, Yonggan, et al.
Published: (2025)

Omni-Recon: Harnessing Image-based Rendering for General-Purpose Neural Radiance Fields
by: Fu, Yonggan, et al.
Published: (2024)

$R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction
by: Du, Zhenbang, et al.
Published: (2026)

TiDAR: Think in Diffusion, Talk in Autoregression
by: Liu, Jingyu, et al.
Published: (2025)

Gen-NeRF: Efficient and Generalizable Neural Radiance Fields via Algorithm-Hardware Co-Design
by: Fu, Yonggan, et al.
Published: (2023)

Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
by: Fu, Yonggan, et al.
Published: (2025)

Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training
by: Diao, Shizhe, et al.
Published: (2025)

Hymba: A Hybrid-head Architecture for Small Language Models
by: Dong, Xin, et al.
Published: (2024)

Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing
by: Fu, Yonggan, et al.
Published: (2022)

Is Mamba Capable of In-Context Learning?
by: Grazzi, Riccardo, et al.
Published: (2024)

Fast-dLLM v2: Efficient Block-Diffusion LLM
by: Wu, Chengyue, et al.
Published: (2025)

NeRFool: Uncovering the Vulnerability of Generalizable Neural Radiance Fields against Adversarial Perturbations
by: Fu, Yonggan, et al.
Published: (2023)

MobileMamba: Lightweight Multi-Receptive Visual Mamba Network
by: He, Haoyang, et al.
Published: (2024)

MambaVision: A Hybrid Mamba-Transformer Vision Backbone
by: Hatamizadeh, Ali, et al.
Published: (2024)

GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Models
by: Fu, Yonggan, et al.
Published: (2023)

Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets
by: You, Haoran, et al.
Published: (2021)

Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search
by: Fu, Yonggan, et al.
Published: (2020)

Robust Tickets Can Transfer Better: Drawing More Transferable Subnetworks in Transfer Learning
by: Fu, Yonggan, et al.
Published: (2023)

MERba: Multi-Receptive Field MambaVision for Micro-Expression Recognition
by: Mao, Xinglong, et al.
Published: (2025)

AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment
by: Fu, Yonggan, et al.
Published: (2024)

ReMamba: Equip Mamba with Effective Long-Sequence Modeling
by: Yuan, Danlong, et al.
Published: (2024)

Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
by: Yu, Zhongzhi, et al.
Published: (2024)

SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation
by: Xing, Zhaohu, et al.
Published: (2024)

Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation
by: Belcak, Peter, et al.
Published: (2025)

AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One
by: Ranzinger, Mike, et al.
Published: (2023)

ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
by: Su, Hongjin, et al.
Published: (2025)

ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge
by: Wang, Zhilin, et al.
Published: (2025)

ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks
by: You, Haoran, et al.
Published: (2022)

A3C-S: Automated Agent Accelerator Co-Search towards Efficient Deep Reinforcement Learning
by: Fu, Yonggan, et al.
Published: (2021)

2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency
by: Fu, Yonggan, et al.
Published: (2021)

Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?
by: Fu, Yonggan, et al.
Published: (2022)

Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators
by: Fu, Yonggan, et al.
Published: (2021)

MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
by: Zhang, Yongan, et al.
Published: (2024)

LongVILA: Scaling Long-Context Visual Language Models for Long Videos
by: Chen, Yukang, et al.
Published: (2024)

CPT: Efficient Deep Neural Network Training via Cyclic Precision
by: Fu, Yonggan, et al.
Published: (2021)

DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
by: Li, Chaojian, et al.
Published: (2021)

Attention Mamba: Time Series Modeling with Adaptive Pooling Acceleration and Receptive Field Enhancements
by: Xiong, Sijie, et al.
Published: (2025)

ZigzagPointMamba: Spatial-Semantic Mamba for Point Cloud Understanding
by: Diao, Linshuang, et al.
Published: (2025)