Image de couverture de livre

How to Compute the Probability of a Word

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Pimentel, Tiago, Meister, Clara
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Towards a Similarity-adjusted Surprisal Theory
par: Meister, Clara, et autres
Publié: (2024)

What Language is This? Ask Your Tokenizer
par: Meister, Clara, et autres
Publié: (2026)

Locally Typical Sampling
par: Meister, Clara, et autres
Publié: (2022)

On the Efficacy of Sampling Adapters
par: Meister, Clara, et autres
Publié: (2023)

Analyzing Wrap-Up Effects through an Information-Theoretic Lens
par: Meister, Clara, et autres
Publié: (2022)

Testing the Predictions of Surprisal Theory in 11 Languages
par: Wilcox, Ethan Gotlieb, et autres
Publié: (2023)

Causal Estimation of Tokenisation Bias
par: Lesci, Pietro, et autres
Publié: (2025)

Back to Bytes: Revisiting Tokenization Through UTF-8
par: Moryossef, Amit, et autres
Publié: (2025)

What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels
par: Yadavalli, Aditya, et autres
Publié: (2025)

Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilities
par: Oh, Byung-Doh, et autres
Publié: (2024)

Formal Aspects of Language Modeling
par: Cotterell, Ryan, et autres
Publié: (2023)

How Should We Model the Probability of a Language?
par: Dent, Rasul, et autres
Publié: (2026)

Probability Distributions Computed by Autoregressive Transformers
par: Yang, Andy, et autres
Publié: (2025)

Tokenisation is NP-Complete
par: Whittington, Philip, et autres
Publié: (2024)

Convergence and Divergence of Language Models under Different Random Seeds
par: Fehlauer, Finlay, et autres
Publié: (2025)

Investigating Critical Period Effects in Language Acquisition through Neural Language Models
par: Constantinescu, Ionut, et autres
Publié: (2024)

Local and Global Decoding in Text Generation
par: Gareev, Daniel, et autres
Publié: (2024)

Position Paper On Diagnostic Uncertainty Estimation from Large Language Models: Next-Word Probability Is Not Pre-test Probability
par: Gao, Yanjun, et autres
Publié: (2024)

Using Letter Positional Probabilities to Assess Word Complexity
par: Dalvean, Michael
Publié: (2024)

Diversidade linguística e inclusão digital: desafios para uma ia brasileira
par: Freitag, Raquel Meister Ko
Publié: (2024)

Uncertainty-Aware Decoding with Minimum Bayes Risk
par: Daheim, Nico, et autres
Publié: (2025)

Investigating Idiomaticity in Word Representations
par: He, Wei, et autres
Publié: (2024)

Speakers Fill Lexical Semantic Gaps with Context
par: Pimentel, Tiago, et autres
Publié: (2020)

The Role of $n$-gram Smoothing in the Age of Neural Networks
par: Malagutti, Luca, et autres
Publié: (2024)

Probing for the Usage of Grammatical Number
par: Lasri, Karim, et autres
Publié: (2022)

How Well Do Large Language Models Disambiguate Swedish Words?
par: Johansson, Richard
Publié: (2024)

How Well Does First-Token Entropy Approximate Word Entropy as a Psycholinguistic Predictor?
par: Clark, Christian, et autres
Publié: (2025)

How Good is BLI as an Alignment Measure: A Study in Word Embedding Paradigm
par: Wickramasinghe, Kasun, et autres
Publié: (2025)

Performance in a dialectal profiling task of LLMs for varieties of Brazilian Portuguese
par: Freitag, Raquel Meister Ko, et autres
Publié: (2024)

An Energy-based Model for Word-level AutoCompletion in Computer-aided Translation
par: Yang, Cheng, et autres
Publié: (2024)

BabyLM's First Words: Word Segmentation as a Phonological Probing Task
par: Goriely, Zébulon, et autres
Publié: (2025)

Simplifications are Absolutists: How Simplified Language Reduces Word Sense Awareness in LLM-Generated Definitions
par: Ellinger, Lukas, et autres
Publié: (2025)

On the Effect of (Near) Duplicate Subwords in Language Modelling
par: Schäfer, Anton, et autres
Publié: (2024)

Tokenisation over Bounded Alphabets is Hard
par: Kastreva, Violeta, et autres
Publié: (2025)

Avaliação de eficiência na leitura: uma abordagem baseada em PLN
par: de Gois, Túlio Sousa, et autres
Publié: (2025)

Is Child-Directed Language Optimized for Word Learning? A Computational Study of Verb Meaning Acquisition
par: Padovani, Francesca, et autres
Publié: (2026)

Tokenisation via Convex Relaxations
par: Tempus, Jan, et autres
Publié: (2026)

AUDDT: Audio Unified Deepfake Detection Benchmark Toolkit
par: Zhu, Yi, et autres
Publié: (2025)

The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations
par: Soler, Aina Garí, et autres
Publié: (2024)

Benchmarking Distributional Alignment of Large Language Models
par: Meister, Nicole, et autres
Publié: (2024)