Image de couverture de livre

Prediction hubs are context-informed frequent tokens in LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Nielsen, Beatrix M. G., Macocco, Iuri, Baroni, Marco
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Not a nuisance but a useful heuristic: Outlier dimensions favor frequent tokens in language models
par: Macocco, Iuri, et autres
Publié: (2025)

Tracing Computation Density in LLMs
par: Kervadec, Corentin, et autres
Publié: (2026)

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
par: Gemini Team, et autres
Publié: (2024)

AnomaLLMy -- Detecting anomalous tokens in black-box LLMs through low-confidence single-token predictions
par: Witold, Waligóra
Publié: (2024)

Jacobian Scopes: token-level causal attributions in LLMs
par: Liu, Toni J. B., et autres
Publié: (2026)

Emergence of a High-Dimensional Abstraction Phase in Language Transformers
par: Cheng, Emily, et autres
Publié: (2024)

Towards Nepali-language LLMs: Efficient GPT training with a Nepali BPE tokenizer
par: Shrestha, Adarsha, et autres
Publié: (2025)

Long-context LLMs Struggle with Long In-context Learning
par: Li, Tianle, et autres
Publié: (2024)

MemoryPrompt: A Light Wrapper to Improve Context Tracking in Pre-trained Language Models
par: Rakotonirina, Nathanaël Carraz, et autres
Publié: (2024)

Interpretable Next-token Prediction via the Generalized Induction Head
par: Kim, Eunji, et autres
Publié: (2024)

Unused information in token probability distribution of generative LLM: improving LLM reading comprehension through calculation of expected values
par: Zawistowski, Krystian
Publié: (2024)

You only need 4 extra tokens: Synergistic Test-time Adaptation for LLMs
par: Xu, Yijie, et autres
Publié: (2025)

Looking beyond the next token
par: Thankaraj, Abitha, et autres
Publié: (2025)

The pitfalls of next-token prediction
par: Bachmann, Gregor, et autres
Publié: (2024)

ThoughtSource: A central hub for large language model reasoning data
par: Ott, Simon, et autres
Publié: (2023)

On the token distance modeling ability of higher RoPE attention dimension
par: Hong, Xiangyu, et autres
Publié: (2024)

Scaled and Inter-token Relation Enhanced Transformer for Sample-restricted Residential NILM
par: Rahman, Minhajur, et autres
Publié: (2024)

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking
par: Hegde, Pradyoth, et autres
Publié: (2025)

No Need for Explanations: LLMs can implicitly learn from mistakes in-context
par: Alazraki, Lisa, et autres
Publié: (2025)

LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
par: Jiang, Ziyan, et autres
Publié: (2024)

Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks
par: Wang, Chonghua, et autres
Publié: (2024)

Shaping capabilities with token-level data filtering
par: Rathi, Neil, et autres
Publié: (2026)

DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysis
par: Georgiou, Efthymios, et autres
Publié: (2025)

SubTokenTest: A Practical Benchmark for Real-World Sub-token Understanding
par: Hou, Shuyang, et autres
Publié: (2026)

Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives
par: Baker, Mohammed Abu, et autres
Publié: (2026)

Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection
par: Miralles-González, Pablo, et autres
Publié: (2025)

Scaling Transformer to 1M tokens and beyond with RMT
par: Bulatov, Aydar, et autres
Publié: (2023)

Evil twins are not that evil: Qualitative insights into machine-generated prompts
par: Rakotonirina, Nathanaël Carraz, et autres
Publié: (2024)

Do LLMs Dream of Ontologies?
par: Bombieri, Marco, et autres
Publié: (2024)

Language models are better than humans at next-token prediction
par: Shlegeris, Buck, et autres
Publié: (2022)

People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text
par: Russell, Jenna, et autres
Publié: (2025)

A Decomposition Perspective to Long-context Reasoning for LLMs
par: Xiao, Yanling, et autres
Publié: (2026)

Meaning Is Not A Metric: Using LLMs to make cultural context legible at scale
par: Kommers, Cody, et autres
Publié: (2025)

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
par: Kwek, Eugene, et autres
Publié: (2025)

TECP: Token-Entropy Conformal Prediction for LLMs
par: Xu, Beining, et autres
Publié: (2025)

Redefining "Hallucination" in LLMs: Towards a psychology-informed framework for mitigating misinformation
par: Berberette, Elijah, et autres
Publié: (2024)

Essential-Web v1.0: 24T tokens of organized web data
par: AI, Essential, et autres
Publié: (2025)

All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling
par: Marconato, Emanuele, et autres
Publié: (2024)

From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMs
par: Chen, Ruxiao, et autres
Publié: (2025)

Harnessing LLMs for Educational Content-Driven Italian Crossword Generation
par: Zeinalipour, Kamyar, et autres
Publié: (2024)