Cover Image

Compressing Sequences in the Latent Embedding Space: $K$-Token Merging for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Zihao, Harvill, John, Fan, Ziwei, Sun, Yizhou, Ding, Hao, Wang, Hao
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Lossless Token Sequence Compression via Meta-Tokens
by: Harvill, John, et al.
Published: (2025)

An Empirical Study on Prompt Compression for Large Language Models
by: Zhang, Zheng, et al.
Published: (2025)

Enhancing Safety of Large Language Models via Embedding Space Separation
by: Zhao, Xu, et al.
Published: (2026)

Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging
by: Liu, Deyuan, et al.
Published: (2024)

Contextual Reinforcement in Multimodal Token Compression for Large Language Models
by: Piero, Naderdel, et al.
Published: (2025)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
by: Shenaj, Donald, et al.
Published: (2025)

A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding
by: Lu, Jinghui, et al.
Published: (2024)

Transport and Merge: Cross-Architecture Merging for Large Language Models
by: Cui, Chenhang, et al.
Published: (2026)

Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization
by: Wang, Dixuan, et al.
Published: (2024)

TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models
by: Yang, Zhen, et al.
Published: (2023)

MAPLE: Mobile App Prediction Leveraging Large Language Model Embeddings
by: Khaokaew, Yonchanok, et al.
Published: (2023)

Hey, That's My Data! Token-Only Dataset Inference in Large Language Models
by: Xiong, Chen, et al.
Published: (2025)

On the Failure of Latent State Persistence in Large Language Models
by: Huang, Jen-tse, et al.
Published: (2025)

A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners
by: Jiang, Bowen, et al.
Published: (2024)

Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

Activation-Informed Merging of Large Language Models
by: Nobari, Amin Heyrani, et al.
Published: (2025)

How Independent are Large Language Models? A Statistical Framework for Auditing Behavioral Entanglement and Reweighting Verifier Ensembles
by: Kuai, Chenchen, et al.
Published: (2026)

Continuous Embedding Attacks via Clipped Inputs in Jailbreaking Large Language Models
by: Xu, Zihao, et al.
Published: (2024)

Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation
by: Zhang, Zhibo, et al.
Published: (2025)

Batching BPE Tokenization Merges
by: Morgan, Alexander P.
Published: (2024)

Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Space
by: Figliolia, Tomas, et al.
Published: (2025)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging
by: Zhao, Yiran, et al.
Published: (2024)

Towards Audio Token Compression in Large Audio Language Models
by: Bhati, Saurabhchand, et al.
Published: (2025)

Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens
by: Zeng, Ziqian, et al.
Published: (2024)

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs
by: Sun, Hao, et al.
Published: (2025)

Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging
by: Lyu, Mengxian, et al.
Published: (2026)

LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
by: Liu, Zehua, et al.
Published: (2025)

LatentBreak: Jailbreaking Large Language Models through Latent Space Feedback
by: Mura, Raffaele, et al.
Published: (2025)

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning
by: Zhang, Tunyu, et al.
Published: (2025)

BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation
by: Pai, Tsung-Min, et al.
Published: (2025)

DPPA: Pruning Method for Large Language Model to Model Merging
by: Zhu, Yaochen, et al.
Published: (2024)

Continual Learning of Large Language Models: A Comprehensive Survey
by: Shi, Haizhou, et al.
Published: (2024)

Active Use of Latent Constituency Representation in both Humans and Large Language Models
by: Liu, Wei, et al.
Published: (2024)

A Survey on Collaborative Mechanisms Between Large and Small Language Models
by: Chen, Yi, et al.
Published: (2025)

Demystifying Embedding Spaces using Large Language Models
by: Tennenholtz, Guy, et al.
Published: (2023)

Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
by: Liu, Peijie, et al.
Published: (2025)

Token Constraint Decoding Improves Robustness on Question Answering for Large Language Models
by: Yao, Jui-Ming, et al.
Published: (2025)

GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy
by: Tan, Hongze, et al.
Published: (2025)