Cover Image

SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Dong, Yu, Yanxuan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

$π$-Attention: Periodic Sparse Transformers for Efficient Long-Context Modeling
by: Liu, Dong, et al.
Published: (2025)

CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference
by: Liu, Dong, et al.
Published: (2025)

Towards Hyper-Efficient RAG Systems in VecDBs: Distributed Parallel Multi-Resolution Vector Search
by: Liu, Dong, et al.
Published: (2025)

Training Text-to-Molecule Models with Context-Aware Tokenization
by: Kim, Seojin, et al.
Published: (2025)

SemShareKV: Efficient KVCache Sharing for Semantically Similar Prompts via Token-Level LSH Matching
by: Zhao, Xinye, et al.
Published: (2025)

Enhancing Large Language Models for Mobility Analytics with Semantic Location Tokenization
by: Chen, Yile, et al.
Published: (2025)

LAWCAT: Efficient Distillation from Quadratic to Linear Attention with Convolution across Tokens for Long Context Modeling
by: Liu, Zeyu, et al.
Published: (2025)

TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
by: Zeng, Runjia, et al.
Published: (2026)

Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
by: Hu, Xiang, et al.
Published: (2025)

QuickMerge++: Fast Token Merging with Autoregressive Prior
by: Liu, Dong, et al.
Published: (2025)

Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning
by: Liu, Dong, et al.
Published: (2026)

Token-Level Uncertainty-Aware Objective for Language Model Post-Training
by: Liu, Tingkai, et al.
Published: (2025)

Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
by: Cao, Qi, et al.
Published: (2026)

Length-MAX Tokenizer for Language Models
by: Dong, Dong, et al.
Published: (2025)

Decoupled Reasoning with Implicit Fact Tokens (DRIFT): A Dual-Model Framework for Efficient Long-Context Inference
by: Xie, Wenxuan, et al.
Published: (2026)

One Token Is Enough: Improving Diffusion Language Models with a Sink Token
by: Zhang, Zihou, et al.
Published: (2026)

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
by: Fu, Qichen, et al.
Published: (2024)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
by: Wu, Wei, et al.
Published: (2024)

Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
by: Liu, Peijie, et al.
Published: (2025)

Thinking Tokens for Language Modeling
by: Herel, David, et al.
Published: (2024)

Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization
by: Wang, Dixuan, et al.
Published: (2024)

Re-Initialization Token Learning for Tool-Augmented Large Language Models
by: Li, Chenghao, et al.
Published: (2025)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
by: Liu, Dong, et al.
Published: (2025)

BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
by: Cao, Qingqing, et al.
Published: (2023)

Semantic Tokens in Retrieval Augmented Generation
by: Suro, Joel
Published: (2024)

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training
by: Qing, Liu, et al.
Published: (2026)

A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
by: Hao, Jitai, et al.
Published: (2025)

Token-Efficient Leverage Learning in Large Language Models
by: Zeng, Yuanhao, et al.
Published: (2024)

Positional Encoding via Token-Aware Phase Attention
by: Wang, Yu, et al.
Published: (2025)

Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
by: Cho, Gyeongje, et al.
Published: (2025)

Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models
by: Yao, Lin
Published: (2026)

Accelerating Multilingual Language Model for Excessively Tokenized Languages
by: Hong, Jimin, et al.
Published: (2024)

Moving Beyond Next-Token Prediction: Transformers are Context-Sensitive Language Generators
by: Rhee, Phill Kyu
Published: (2025)

Context-Aware Semantic Recomposition Mechanism for Large Language Models
by: Katrix, Richard, et al.
Published: (2025)

Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks
by: Xu, Tianze, et al.
Published: (2026)

STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Models
by: Sun, Xinhao, et al.
Published: (2025)

HSGM: Hierarchical Segment-Graph Memory for Scalable Long-Text Semantics
by: Liu, Dong, et al.
Published: (2025)

CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
by: Shah, Deep, et al.
Published: (2026)

SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning
by: Li, Zheng, et al.
Published: (2025)

Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understanding
by: Zhang, Zhongjian, et al.
Published: (2026)