Cover Image

Rethinking Token Reduction for State Space Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhan, Zheng, Wu, Yushu, Kong, Zhenglun, Yang, Changdi, Gong, Yifan, Shen, Xuan, Lin, Xue, Zhao, Pu, Wang, Yanzhi
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Exploring Token Pruning in Vision State Space Models
by: Zhan, Zheng, et al.
Published: (2024)

Squat: Quant Small Language Models on the Edge
by: Shen, Xuan, et al.
Published: (2024)

Pruning Foundation Models for High Accuracy without Retraining
by: Zhao, Pu, et al.
Published: (2024)

Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment
by: Liu, Jun, et al.
Published: (2024)

Search for Efficient Large Language Models
by: Shen, Xuan, et al.
Published: (2024)

Fast and Memory-Efficient Video Diffusion Using Streamlined Inference
by: Zhan, Zheng, et al.
Published: (2024)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)

Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
by: Shen, Xuan, et al.
Published: (2023)

TSLA: A Task-Specific Learning Adaptation for Semantic Segmentation on Autonomous Vehicles Platform
by: Liu, Jun, et al.
Published: (2025)

Enabling Flexible Multi-LLM Integration for Scalable Knowledge Aggregation
by: Kong, Zhenglun, et al.
Published: (2025)

RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
by: Liu, Jun, et al.
Published: (2025)

Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA
by: Zhao, Pu, et al.
Published: (2025)

7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement
by: Zhao, Pu, et al.
Published: (2024)

Lotus: learning-based online thermal and latency variation management for two-stage detectors on edge devices
by: Gong, Yifan, et al.
Published: (2024)

Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality
by: Kong, Zhenglun, et al.
Published: (2025)

OmniMem: Scalable and Adaptive Memory Retrieval for Long Video Generation
by: Zhao, Lin, et al.
Published: (2026)

RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory
by: Liu, Jun, et al.
Published: (2025)

Rethinking Token Prediction: Tree-Structured Diffusion Language Model
by: Wu, Zihao, et al.
Published: (2026)

Efficient Reasoning with Hidden Thinking
by: Shen, Xuan, et al.
Published: (2025)

ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Models
by: Zheng, Kangjie, et al.
Published: (2025)

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
by: Zhan, Zheng, et al.
Published: (2025)

Token Hidden Reward: Steering Exploration-Exploitation in Group Relative Deep Reinforcement Learning
by: Deng, Wenlong, et al.
Published: (2025)

MambaByte: Token-free Selective State Space Model
by: Wang, Junxiong, et al.
Published: (2024)

DiffClass: Diffusion-Based Class Incremental Learning
by: Meng, Zichong, et al.
Published: (2024)

Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models
by: Zhao, Dachuan, et al.
Published: (2025)

Taming Diffusion for Dataset Distillation with High Representativeness
by: Zhao, Lin, et al.
Published: (2025)

MemMamba: Rethinking Memory Patterns in State Space Model
by: Wang, Youjin, et al.
Published: (2025)

EvoMerge: Neuroevolution for Large Language Models
by: Jiang, Yushu
Published: (2024)

Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
by: Shen, Yangyi, et al.
Published: (2026)

DynaAct: Large Language Model Reasoning with Dynamic Action Spaces
by: Zhao, Xueliang, et al.
Published: (2025)

Overcoming Long-Context Limitations of State-Space Models via Context-Dependent Sparse Attention
by: Zhan, Zhihao, et al.
Published: (2025)

When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making
by: Liu, Jun, et al.
Published: (2026)

Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
by: Ren, Liliang, et al.
Published: (2024)

One Pass Streaming Algorithm for Super Long Token Attention Approximation in Sublinear Space
by: Addanki, Raghav, et al.
Published: (2023)

PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning
by: Zhao, Xueliang, et al.
Published: (2025)

Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction
by: Shi, Zhenmei, et al.
Published: (2024)

Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models
by: Ghiasvand, Sajjad, et al.
Published: (2024)

Verbal Werewolf: Engage Users with Verbalized Agentic Werewolf Game Framework
by: Fan, Qihui, et al.
Published: (2025)

Rethinking Personalization in Large Language Models at the Token Level
by: Zhang, Chenheng, et al.
Published: (2026)

Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models
by: Yang, Jinbiao
Published: (2024)