Imagen de Portada

No Clustering, No Routing: How Transformers Actually Process Rare Tokens

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Liu, Jing
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
por: Vakilian, Vala, et al.
Publicado: (2025)

An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
por: Chai, Ziwei, et al.
Publicado: (2024)

STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens
por: Liu, Shiqi, et al.
Publicado: (2026)

Beyond Questions: Evaluating What Large Language Models (Actually) Know
por: Giordano, Luca, et al.
Publicado: (2026)

NLP Methods May Actually Be Better Than Professors at Estimating Question Difficulty
por: Zotos, Leonidas, et al.
Publicado: (2025)

Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?
por: Lee, Chia-Hsuan, et al.
Publicado: (2026)

Learning to Route LLMs with Confidence Tokens
por: Chuang, Yu-Neng, et al.
Publicado: (2024)

Informed Routing in LLMs: Smarter Token-Level Computation for Faster Inference
por: Han, Chao, et al.
Publicado: (2025)

MoLoRA: Composable Specialization via Per-Token Adapter Routing
por: Shah, Shrey, et al.
Publicado: (2026)

SliceMoE: Routing Embedding Slices Instead of Tokens for Fine-Grained and Balanced Transformer Scaling
por: Vejendla, Harshil
Publicado: (2025)

How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing
por: Marín, Javier
Publicado: (2026)

Rubric-Guided Process Reward for Stepwise Model Routing
por: Ye, Shenghao, et al.
Publicado: (2026)

Token-Level LLM Collaboration via FusionRoute
por: Xiong, Nuoya, et al.
Publicado: (2026)

Token Masking Improves Transformer-Based Text Classification
por: Xu, Xianglong, et al.
Publicado: (2025)

Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines
por: Jørgensen, Mikkel Godsk, et al.
Publicado: (2026)

Advanced Unstructured Data Processing for ESG Reports: A Methodology for Structured Transformation and Enhanced Analysis
por: Peng, Jiahui, et al.
Publicado: (2024)

Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
por: Ren, Richard, et al.
Publicado: (2024)

TARo: Token-level Adaptive Routing for LLM Test-time Alignment
por: Rai, Arushi, et al.
Publicado: (2026)

Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning
por: Qian, Chen, et al.
Publicado: (2025)

KunlunBaize: LLM with Multi-Scale Convolution and Multi-Token Prediction Under TransformerX Framework
por: Li, Cheng, et al.
Publicado: (2025)

SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
por: Liu, Dong, et al.
Publicado: (2025)

How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning
por: Chen, Haoyang, et al.
Publicado: (2026)

Transformer-based Causal Language Models Perform Clustering
por: Wu, Xinbo, et al.
Publicado: (2024)

Moving Beyond Next-Token Prediction: Transformers are Context-Sensitive Language Generators
por: Rhee, Phill Kyu
Publicado: (2025)

Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?
por: Sakai, Yusuke, et al.
Publicado: (2023)

MemRouter: Memory-as-Embedding Routing for Long-Term Conversational Agents
por: Hu, Tianyu, et al.
Publicado: (2026)

Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations
por: Bochkov, A.
Publicado: (2025)

Uncovering the Role of Initial Saliency in U-Shaped Attention Bias: Scaling Initial Token Weight for Enhanced Long-Text Processing
por: Qiang, Zewen, et al.
Publicado: (2025)

Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
por: Liu, Peijie, et al.
Publicado: (2025)

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
por: Zheng, Wenhao, et al.
Publicado: (2025)

One Token Is Enough: Improving Diffusion Language Models with a Sink Token
por: Zhang, Zihou, et al.
Publicado: (2026)

The Tokenization Bottleneck: How Vocabulary Extension Improves Chemistry Representation Learning in Pretrained Language Models
por: Kalamkar, Prathamesh, et al.
Publicado: (2025)

How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach
por: Lee, Ayeong, et al.
Publicado: (2025)

S$^2$-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency
por: Zeng, Yuting, et al.
Publicado: (2025)

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training
por: Qing, Liu, et al.
Publicado: (2026)

The Token Tax: Systematic Bias in Multilingual Tokenization
por: Lundin, Jessica M., et al.
Publicado: (2025)

How Different Tokenization Algorithms Impact LLMs and Transformer Models for Binary Code Analysis
por: Mostafa, Ahmed, et al.
Publicado: (2025)

Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
por: Cao, Qi, et al.
Publicado: (2026)

TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
por: Zeng, Runjia, et al.
Publicado: (2026)

How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study
por: Zhu, Shuqi, et al.
Publicado: (2026)