Cover Image

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Wei, Han, Kai, Tang, Yehui, Wang, Chengcheng, Yang, Yujie, Guo, Tianyu, Wang, Yunhe
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models
by: Ni, Yunsheng, et al.
Published: (2024)

PocketLLM: Ultimate Compression of Large Language Models via Meta Networks
by: Tian, Ye, et al.
Published: (2025)

Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuning
by: Jie, Shibo, et al.
Published: (2024)

Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs
by: Yin, Yichun, et al.
Published: (2025)

Saliency-driven Dynamic Token Pruning for Large Language Models
by: Tao, Yao, et al.
Published: (2025)

SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution
by: Wang, Chengcheng, et al.
Published: (2024)

Making Large Language Models Efficient Dense Retrievers
by: Lei, Yibin, et al.
Published: (2025)

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization
by: Guo, Jialong, et al.
Published: (2024)

DenseLoRA: Dense Low-Rank Adaptation of Large Language Models
by: Mu, Lin, et al.
Published: (2025)

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
by: Bi, Zhenni, et al.
Published: (2024)

MLP Fusion: Towards Efficient Fine-tuning of Dense and Mixture-of-Experts Language Models
by: Ai, Mengting, et al.
Published: (2023)

Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning
by: Zhou, Hang, et al.
Published: (2024)

Multi-Granularity Semantic Revision for Large Language Model Distillation
by: Liu, Xiaoyu, et al.
Published: (2024)

SlimLLM: Accurate Structured Pruning for Large Language Models
by: Guo, Jialong, et al.
Published: (2025)

Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval
by: Liu, Yuxiang, et al.
Published: (2025)

MemoryFormer: Minimize Transformer Computation by Removing Fully-Connected Layers
by: Ding, Ning, et al.
Published: (2024)

From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan
by: Yang, Lei, et al.
Published: (2025)

Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting
by: Liu, Fangcheng, et al.
Published: (2024)

CBQ: Cross-Block Quantization for Large Language Models
by: Ding, Xin, et al.
Published: (2023)

Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts
by: Rang, Miao, et al.
Published: (2025)

ML-Mamba: Efficient Multi-Modal Large Language Model Utilizing Mamba-2
by: Huang, Wenjun, et al.
Published: (2024)

CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models
by: Nie, Ying, et al.
Published: (2024)

A Survey on Transformer Compression
by: Tang, Yehui, et al.
Published: (2024)

Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Models
by: Peng, Zhiyuan, et al.
Published: (2023)

Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs
by: Han, Kai, et al.
Published: (2024)

DiJiang: Efficient Large Language Models through Compact Kernelization
by: Chen, Hanting, et al.
Published: (2024)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

PanGu-$π$ Pro:Rethinking Optimization and Architecture for Tiny Language Models
by: Tang, Yehui, et al.
Published: (2024)

Exploring the Capabilities of Large Multimodal Models on Dense Text
by: Zhang, Shuo, et al.
Published: (2024)

Hidden State Poisoning Attacks against Mamba-based Language Models
by: Mercier, Alexandre Le, et al.
Published: (2026)

EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
by: Fu, Zhongqian, et al.
Published: (2025)

Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrieval
by: Liu, Zheng, et al.
Published: (2023)

Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft
by: Li, Hao, et al.
Published: (2023)

Unshackling Context Length: An Efficient Selective Attention Approach through Query-Key Compression
by: Wang, Haoyu, et al.
Published: (2025)

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
by: Feng, Yuchen, et al.
Published: (2025)

Self-Improvement in Multimodal Large Language Models: A Survey
by: Deng, Shijian, et al.
Published: (2025)

PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation
by: Wang, Yunhe, et al.
Published: (2023)

DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers
by: Ma, Xueguang, et al.
Published: (2025)

SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs
by: Jie, Shibo, et al.
Published: (2025)

A robust audio deepfake detection system via multi-view feature
by: Yang, Yujie, et al.
Published: (2024)