Cover Image

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kwek, Eugene, Yin, Wenpeng
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Looking beyond the next token
by: Thankaraj, Abitha, et al.
Published: (2025)

The pitfalls of next-token prediction
by: Bachmann, Gregor, et al.
Published: (2024)

SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning
by: Liao, Huanxuan, et al.
Published: (2025)

Think Clearly: Improving Reasoning via Redundant Token Pruning
by: Choi, Daewon, et al.
Published: (2025)

Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference
by: Taniguchi, Rei, et al.
Published: (2026)

Scaling Optimal LR Across Token Horizons
by: Bjorck, Johan, et al.
Published: (2024)

Shaping capabilities with token-level data filtering
by: Rathi, Neil, et al.
Published: (2026)

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
by: Fu, Qichen, et al.
Published: (2024)

All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling
by: Marconato, Emanuele, et al.
Published: (2024)

LeanK: Learnable K Cache Channel Pruning for Efficient Decoding
by: Zhang, Yike, et al.
Published: (2025)

Scaling Transformer to 1M tokens and beyond with RMT
by: Bulatov, Aydar, et al.
Published: (2023)

Interpretable Next-token Prediction via the Generalized Induction Head
by: Kim, Eunji, et al.
Published: (2024)

Language models are better than humans at next-token prediction
by: Shlegeris, Buck, et al.
Published: (2022)

Understanding Dynamic Compute Allocation in Recurrent Transformers
by: Moosa, Ibraheem Muhammad, et al.
Published: (2026)

Direct-Inverse Prompting: Analyzing LLMs' Discriminative Capacity in Self-Improving Generation
by: Ahn, Jihyun Janice, et al.
Published: (2024)

The Geometries of Truth Are Orthogonal Across Tasks
by: Azizian, Waiss, et al.
Published: (2025)

Large Language Model Pruning
by: Huang, Hanjuan, et al.
Published: (2024)

IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
by: Li, Yixiao, et al.
Published: (2025)

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing
by: Le, Qi, et al.
Published: (2025)

Essential-Web v1.0: 24T tokens of organized web data
by: AI, Essential, et al.
Published: (2025)

CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models
by: Xiao, Xiaojun, et al.
Published: (2024)

Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models
by: Wang, Ziyan, et al.
Published: (2025)

You only need 4 extra tokens: Synergistic Test-time Adaptation for LLMs
by: Xu, Yijie, et al.
Published: (2025)

Sink-Aware Pruning for Diffusion Language Models
by: Myrzakhan, Aidar, et al.
Published: (2026)

UniMoT: Unified Molecule-Text Language Model with Discrete Token Representation
by: Guo, Shuhan, et al.
Published: (2024)

ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimization
by: Yoon, Hee Suk, et al.
Published: (2025)

TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models
by: Choo, Jinho, et al.
Published: (2026)

T-REG: Preference Optimization with Token-Level Reward Regularization
by: Zhou, Wenxuan, et al.
Published: (2024)

DPO Meets PPO: Reinforced Token Optimization for RLHF
by: Zhong, Han, et al.
Published: (2024)

Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction
by: Nagarajan, Vaishnavh, et al.
Published: (2025)

Is Sanskrit the most token-efficient language? A quantitative study using GPT, Gemini, and SentencePiece
by: Kumar, Anshul
Published: (2026)

Contrastive Instruction Tuning
by: Yan, Tianyi Lorena, et al.
Published: (2024)

Beware of Calibration Data for Pruning Large Language Models
by: Ji, Yixin, et al.
Published: (2024)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
by: Kim, Junhyuck, et al.
Published: (2026)

Pruning Foundation Models for High Accuracy without Retraining
by: Zhao, Pu, et al.
Published: (2024)

COPAL: Continual Pruning in Large Language Generative Models
by: Malla, Srikanth, et al.
Published: (2024)

Compact Language Models via Pruning and Knowledge Distillation
by: Muralidharan, Saurav, et al.
Published: (2024)

Lightweight Safety Classification Using Pruned Language Models
by: Sawtell, Mason, et al.
Published: (2024)

PAT: Pruning-Aware Tuning for Large Language Models
by: Liu, Yijiang, et al.
Published: (2024)

Long Chain-of-Thought Reasoning Across Languages
by: Barua, Josh, et al.
Published: (2025)