Image de couverture de livre

No Clustering, No Routing: How Transformers Actually Process Rare Tokens

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Liu, Jing
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
par: Vakilian, Vala, et autres
Publié: (2025)

An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
par: Chai, Ziwei, et autres
Publié: (2024)

STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens
par: Liu, Shiqi, et autres
Publié: (2026)

Beyond Questions: Evaluating What Large Language Models (Actually) Know
par: Giordano, Luca, et autres
Publié: (2026)

NLP Methods May Actually Be Better Than Professors at Estimating Question Difficulty
par: Zotos, Leonidas, et autres
Publié: (2025)

Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?
par: Lee, Chia-Hsuan, et autres
Publié: (2026)

Learning to Route LLMs with Confidence Tokens
par: Chuang, Yu-Neng, et autres
Publié: (2024)

Informed Routing in LLMs: Smarter Token-Level Computation for Faster Inference
par: Han, Chao, et autres
Publié: (2025)

MoLoRA: Composable Specialization via Per-Token Adapter Routing
par: Shah, Shrey, et autres
Publié: (2026)

SliceMoE: Routing Embedding Slices Instead of Tokens for Fine-Grained and Balanced Transformer Scaling
par: Vejendla, Harshil
Publié: (2025)

How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing
par: Marín, Javier
Publié: (2026)

Rubric-Guided Process Reward for Stepwise Model Routing
par: Ye, Shenghao, et autres
Publié: (2026)

Token-Level LLM Collaboration via FusionRoute
par: Xiong, Nuoya, et autres
Publié: (2026)

Token Masking Improves Transformer-Based Text Classification
par: Xu, Xianglong, et autres
Publié: (2025)

Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines
par: Jørgensen, Mikkel Godsk, et autres
Publié: (2026)

Advanced Unstructured Data Processing for ESG Reports: A Methodology for Structured Transformation and Enhanced Analysis
par: Peng, Jiahui, et autres
Publié: (2024)

Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
par: Ren, Richard, et autres
Publié: (2024)

TARo: Token-level Adaptive Routing for LLM Test-time Alignment
par: Rai, Arushi, et autres
Publié: (2026)

Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning
par: Qian, Chen, et autres
Publié: (2025)

KunlunBaize: LLM with Multi-Scale Convolution and Multi-Token Prediction Under TransformerX Framework
par: Li, Cheng, et autres
Publié: (2025)

SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
par: Liu, Dong, et autres
Publié: (2025)

How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning
par: Chen, Haoyang, et autres
Publié: (2026)

Transformer-based Causal Language Models Perform Clustering
par: Wu, Xinbo, et autres
Publié: (2024)

Moving Beyond Next-Token Prediction: Transformers are Context-Sensitive Language Generators
par: Rhee, Phill Kyu
Publié: (2025)

Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?
par: Sakai, Yusuke, et autres
Publié: (2023)

MemRouter: Memory-as-Embedding Routing for Long-Term Conversational Agents
par: Hu, Tianyu, et autres
Publié: (2026)

Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations
par: Bochkov, A.
Publié: (2025)

Uncovering the Role of Initial Saliency in U-Shaped Attention Bias: Scaling Initial Token Weight for Enhanced Long-Text Processing
par: Qiang, Zewen, et autres
Publié: (2025)

Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
par: Liu, Peijie, et autres
Publié: (2025)

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
par: Zheng, Wenhao, et autres
Publié: (2025)

One Token Is Enough: Improving Diffusion Language Models with a Sink Token
par: Zhang, Zihou, et autres
Publié: (2026)

The Tokenization Bottleneck: How Vocabulary Extension Improves Chemistry Representation Learning in Pretrained Language Models
par: Kalamkar, Prathamesh, et autres
Publié: (2025)

How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach
par: Lee, Ayeong, et autres
Publié: (2025)

S$^2$-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency
par: Zeng, Yuting, et autres
Publié: (2025)

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training
par: Qing, Liu, et autres
Publié: (2026)

The Token Tax: Systematic Bias in Multilingual Tokenization
par: Lundin, Jessica M., et autres
Publié: (2025)

How Different Tokenization Algorithms Impact LLMs and Transformer Models for Binary Code Analysis
par: Mostafa, Ahmed, et autres
Publié: (2025)

Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
par: Cao, Qi, et autres
Publié: (2026)

TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
par: Zeng, Runjia, et autres
Publié: (2026)

How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study
par: Zhu, Shuqi, et autres
Publié: (2026)