Image de couverture de livre

Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Prins, Zoë, Punzo, Samuele, Wildenburg, Frank, Cinà, Giovanni, Pezzelle, Sandro
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Do Pre-Trained Language Models Detect and Understand Semantic Underspecification? Ask the DUST!
par: Wildenburg, Frank, et autres
Publié: (2024)

Naming, Describing, and Quantifying Visual Objects in Humans and LLMs
par: Testoni, Alberto, et autres
Publié: (2024)

They want to pretend not to understand: The Limits of Current LLMs in Interpreting Implicit Content of Political Discourse
par: Paci, Walter, et autres
Publié: (2025)

The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models
par: Chen, Xinyi, et autres
Publié: (2023)

Are formal and functional linguistic mechanisms dissociated in language models?
par: Hanna, Michael, et autres
Publié: (2025)

Who is the richest club in the championship? Detecting and Rewriting Underspecified Questions Improve QA Performance
par: Huang, Yunchong, et autres
Publié: (2026)

Beyond Divergent Creativity: A Human-Based Evaluation of Creativity in Large Language Models
par: Nakajima, Kumiko, et autres
Publié: (2026)

How Language Models Conflate Logical Validity with Plausibility: A Representational Analysis of Content Effects
par: Bertolazzi, Leonardo, et autres
Publié: (2025)

Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information
par: Hu, Zhengmian, et autres
Publié: (2023)

Describing Images $\textit{Fast and Slow}$: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processes
par: Takmaz, Ece, et autres
Publié: (2024)

Rethinking Perplexity: Revealing the Impact of Input Length on Perplexity Evaluation in LLMs
par: Cheng, Letian, et autres
Publié: (2026)

Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms
par: Hanna, Michael, et autres
Publié: (2024)

WatChat: Explaining perplexing programs by debugging mental models
par: Chandra, Kartik, et autres
Publié: (2024)

Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Contextually Important Moments
par: Surikuchi, Aditya K, et autres
Publié: (2026)

Is my model "mind blurting"? Interpreting the dynamics of reasoning tokens with Recurrence Quantification Analysis (RQA)
par: Pham, Quoc Tuan, et autres
Publié: (2026)

Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetition
par: Surikuchi, Aditya K, et autres
Publié: (2024)

Natural Language Generation from Visual Events: State-of-the-Art and Key Open Questions
par: Surikuchi, Aditya K, et autres
Publié: (2025)

On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation
par: Hsu, Chan-Jan, et autres
Publié: (2026)

Vision-Language Models Align with Human Neural Representations in Concept Processing
par: Bavaresco, Anna, et autres
Publié: (2024)

Mapping Overlaps in Benchmarks through Perplexity in the Wild
par: Wu, Siyang, et autres
Publié: (2025)

Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models
par: Ankner, Zachary, et autres
Publié: (2024)

Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learning
par: Wu, Chao-Chung, et autres
Publié: (2025)

Confidence, Not Perplexity: A Better Metric for the Creative Era of LLMs
par: Parupudi, V. S. Raghu
Publié: (2025)

Distributional reasoning in LLMs: Parallel reasoning processes in multi-hop reasoning
par: Shalev, Yuval, et autres
Publié: (2024)

Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization
par: Mavromatis, Costas, et autres
Publié: (2024)

Inconsistent Tokenizations Cause Language Models to be Perplexed by Japanese Grammar
par: Gambardella, Andrew, et autres
Publié: (2025)

Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models
par: Klein, Tassilo, et autres
Publié: (2024)

From Tools to Teammates: Evaluating LLMs in Multi-Session Coding Interactions
par: Rakotonirina, Nathanaël Carraz, et autres
Publié: (2025)

Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs
par: Dong, Jiancheng, et autres
Publié: (2025)

How Well Can Knowledge Edit Methods Edit Perplexing Knowledge?
par: Ge, Huaizhi, et autres
Publié: (2024)

Beyond Perplexity: Character Distribution Signatures and the MDTA Benchmark for AI Text Detection
par: Narayanasamy, Priyadarshan, et autres
Publié: (2026)

QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?
par: Li, Belinda Z., et autres
Publié: (2025)

Beyond Tokens: Concept-Level Training Objectives for LLMs
par: Iyer, Laya, et autres
Publié: (2026)

When Meaning Stays the Same, but Models Drift: Evaluating Quality of Service under Token-Level Behavioral Instability in LLMs
par: Li, Xiao, et autres
Publié: (2025)

Can LLMs Simulate Human Behavioral Variability? A Case Study in the Phonemic Fluency Task
par: Qiu, Mengyang, et autres
Publié: (2025)

Beyond Perplexity: A Lightweight Benchmark for Knowledge Retention in Supervised Fine-Tuning
par: Shabgahi, Soheil Zibakhsh, et autres
Publié: (2026)

Can Perplexity Predict Fine-tuning Performance? An Investigation of Tokenization Effects on Sequential Language Models for Nepali
par: Luitel, Nishant, et autres
Publié: (2024)

Rectify Evaluation Preference: Improving LLMs' Critique on Math Reasoning via Perplexity-aware Reinforcement Learning
par: Tian, Changyuan, et autres
Publié: (2025)

Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documents
par: Wang, Haoyu, et autres
Publié: (2025)

Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training
par: Liu, Lei, et autres
Publié: (2025)