Copertina

How Well Does First-Token Entropy Approximate Word Entropy as a Psycholinguistic Predictor?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Clark, Christian, Oh, Byung-Doh, Schuler, William
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilities
di: Oh, Byung-Doh, et al.
Pubblicazione: (2024)

The Impact of Token Granularity on the Predictive Power of Language Model Surprisal
di: Oh, Byung-Doh, et al.
Pubblicazione: (2024)

Linear Recency Bias During Training Improves Transformers' Fit to Reading Times
di: Clark, Christian, et al.
Pubblicazione: (2024)

The Inverse Scaling Effect of Pre-Trained Language Model Surprisal Is Not Due to Data Leakage
di: Oh, Byung-Doh, et al.
Pubblicazione: (2025)

Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Times
di: Oh, Byung-Doh, et al.
Pubblicazione: (2024)

Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal
di: Nair, Sathvik, et al.
Pubblicazione: (2026)

On the Proper Treatment of Tokenization in Psycholinguistics
di: Giulianelli, Mario, et al.
Pubblicazione: (2024)

To Words and Beyond: Probing Large Language Models for Sentence-Level Psycholinguistic Norms of Memorability and Reading Times
di: Clark, Thomas Hikaru, et al.
Pubblicazione: (2026)

LanguaShrink: Reducing Token Overhead with Psycholinguistics
di: Liang, Xuechen, et al.
Pubblicazione: (2024)

Psycholinguistic Word Features: a New Approach for the Evaluation of LLMs Alignment with Humans
di: Conde, Javier, et al.
Pubblicazione: (2025)

EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
di: Cheong, Minsoo, et al.
Pubblicazione: (2026)

Entropy-Driven Pre-Tokenization for Byte-Pair Encoding
di: Hu, Yifan, et al.
Pubblicazione: (2025)

TECP: Token-Entropy Conformal Prediction for LLMs
di: Xu, Beining, et al.
Pubblicazione: (2025)

EGAD: Entropy-Guided Adaptive Distillation for Token-Level Knowledge Transfer
di: Zhang, Hao, et al.
Pubblicazione: (2026)

How Well Do Large Language Models Disambiguate Swedish Words?
di: Johansson, Richard
Pubblicazione: (2024)

Token-Level Entropy Reveals Demographic Disparities in Language Models
di: Lee, Messi H. J.
Pubblicazione: (2025)

Large Language Models for Psycholinguistic Plausibility Pretesting
di: Amouyal, Samuel Joseph, et al.
Pubblicazione: (2024)

A Survey of Idiom Datasets for Psycholinguistic and Computational Research
di: Flor, Michael, et al.
Pubblicazione: (2025)

ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping
di: Chen, Shuang, et al.
Pubblicazione: (2025)

Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data
di: Wang, Jiapeng, et al.
Pubblicazione: (2025)

Not All Tokens Learn Alike: Attention Entropy Reveals Heterogeneous Signals in RL Reasoning
di: Li, Gengyang, et al.
Pubblicazione: (2026)

GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy
di: Tan, Hongze, et al.
Pubblicazione: (2025)

ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs
di: Chen, Xunlei, et al.
Pubblicazione: (2026)

Entropy of Ukrainian
di: Lavreniuk, Anton, et al.
Pubblicazione: (2026)

A Psycholinguistic Evaluation of Language Models' Sensitivity to Argument Roles
di: Lee, Eun-Kyoung Rosa, et al.
Pubblicazione: (2024)

SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularization
di: Wu, Jianghao, et al.
Pubblicazione: (2025)

Unveiling Language Competence Neurons: A Psycholinguistic Approach to Model Interpretability
di: Duan, Xufeng, et al.
Pubblicazione: (2024)

A Temporal Psycholinguistics Approach to Identity Resolution of Social Media Users
di: Islam, Md Touhidul
Pubblicazione: (2024)

Two-step Automated Cybercrime Coded Word Detection using Multi-level Representation Learning
di: Kim, Yongyeon, et al.
Pubblicazione: (2024)

Psychological Profiling in Cybersecurity: A Look at LLMs and Psycholinguistic Features
di: Tshimula, Jean Marie, et al.
Pubblicazione: (2024)

Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Models
di: Wu, Jialiang, et al.
Pubblicazione: (2025)

Subword Tokenization Strategies for Kurdish Word Embeddings
di: Salehi, Ali, et al.
Pubblicazione: (2025)

Entropy Sentinel: Continuous LLM Accuracy Monitoring from Decoding Entropy Traces in STEM
di: Buffa, Pedro Memoli, et al.
Pubblicazione: (2026)

Stop Jostling: Adaptive Negative Sampling Reduces the Marginalization of Low-Resource Language Tokens by Cross-Entropy Loss
di: Turumtaev, Galim
Pubblicazione: (2026)

Entropy-Tree: Tree-Based Decoding with Entropy-Guided Exploration
di: Wei, Longxuan, et al.
Pubblicazione: (2026)

Probing Geometry of Next Token Prediction Using Cumulant Expansion of the Softmax Entropy
di: Viswanathan, Karthik, et al.
Pubblicazione: (2025)

Reasoning with Exploration: An Entropy Perspective
di: Cheng, Daixuan, et al.
Pubblicazione: (2025)

One-shot Entropy Minimization
di: Gao, Zitian, et al.
Pubblicazione: (2025)

Entropy-Guided Reasoning Compression
di: Zhu, Hourun, et al.
Pubblicazione: (2025)

Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Rate
di: Moslonka, Charles, et al.
Pubblicazione: (2025)