Image de couverture de livre

Knowledge of Pretrained Language Models on Surface Information of Tokens

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Hiraoka, Tatsuya, Okazaki, Naoaki
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Bit-level BPE: Below the byte boundary
par: Moon, Sangwhan, et autres
Publié: (2025)

An Analysis of BPE Vocabulary Trimming in Neural Machine Translation
par: Cognetta, Marco, et autres
Publié: (2024)

Tokenization as Finite-State Transduction
par: Cognetta, Marco, et autres
Publié: (2024)

Tokenization Preference for Human and Machine Learning Model: An Annotation Study
par: Hiraoka, Tatsuya, et autres
Publié: (2023)

Spelling-out is not Straightforward: LLMs' Capability of Tokenization from Token to Characters
par: Hiraoka, Tatsuya, et autres
Publié: (2025)

From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models
par: Ma, Youmi, et autres
Publié: (2026)

Repetition Neurons: How Do Language Models Produce Repetitions?
par: Hiraoka, Tatsuya, et autres
Publié: (2024)

Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels
par: Anantaprayoon, Panatchakorn, et autres
Publié: (2023)

Two Counterexamples to Tokenization and the Noiseless Channel
par: Cognetta, Marco, et autres
Publié: (2024)

Social Bias Evaluation for Large Language Models Requires Prompt Variations
par: Hida, Rem, et autres
Publié: (2024)

Drifting Objectives for Refining Discrete Diffusion Language Models
par: Oba, Daisuke, et autres
Publié: (2026)

Diffusion-State Policy Optimization for Masked Diffusion Language Models
par: Oba, Daisuke, et autres
Publié: (2026)

Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs
par: Miyamoto, Sora, et autres
Publié: (2026)

Intent-Aware Self-Correction for Mitigating Social Biases in Large Language Models
par: Anantaprayoon, Panatchakorn, et autres
Publié: (2025)

Stopping Computation for Converged Tokens in Masked Diffusion-LM Decoding
par: Oba, Daisuke, et autres
Publié: (2026)

Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting
par: Kaneko, Masahiro, et autres
Publié: (2024)

Synthesizing Instruction-Tuning Datasets with Contrastive Decoding
par: Ichinose, Tatsuya, et autres
Publié: (2026)

Building a Japanese Document-Level Relation Extraction Dataset Assisted by Cross-Lingual Transfer
par: Ma, Youmi, et autres
Publié: (2024)

Distributional Properties of Subword Regularization
par: Cognetta, Marco, et autres
Publié: (2024)

A Japanese Benchmark for Evaluating Social Bias in Reasoning Based on Attribution Theory
par: Shiotani, Taihei, et autres
Publié: (2026)

OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples
par: Koike, Ryuto, et autres
Publié: (2023)

How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection
par: Koike, Ryuto, et autres
Publié: (2023)

Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach
par: Kaneko, Masahiro, et autres
Publié: (2023)

SAIE Framework: Support Alone Isn't Enough -- Advancing LLM Training with Adversarial Remarks
par: Loem, Mengsay, et autres
Publié: (2023)

Multi-modal, Multi-task, Multi-criteria Automatic Evaluation with Vision Language Models
par: Ohi, Masanari, et autres
Publié: (2024)

Likelihood-based Mitigation of Evaluation Bias in Large Language Models
par: Oi, Masanari, et autres
Publié: (2024)

Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model
par: Sasagawa, Keito, et autres
Publié: (2024)

Predicting the Emergence of Induction Heads in Language Model Pretraining
par: Aoyama, Tatsuya, et autres
Publié: (2025)

Sampling-based Pseudo-Likelihood for Membership Inference Attacks
par: Kaneko, Masahiro, et autres
Publié: (2024)

LLM Output Detectability and Task Performance Can be Jointly Optimized
par: Saito, Koshiro, et autres
Publié: (2026)

WAON: A Large-Scale Japanese Image-Text Dataset for Cultural Adaptation in Contrastive Vision-Language Models
par: Sugiura, Issa, et autres
Publié: (2025)

Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning
par: Doan, Nhi Hoai, et autres
Publié: (2025)

The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
par: El-Shangiti, Ahmed Oumar, et autres
Publié: (2024)

Decoding-Free Sampling Strategies for LLM Marginalization
par: Pohl, David, et autres
Publié: (2025)

ExaGPT: Example-Based Machine-Generated Text Detection for Human Interpretability
par: Koike, Ryuto, et autres
Publié: (2025)

SubRegWeigh: Effective and Efficient Annotation Weighing with Subword Regularization
par: Tsuji, Kohei, et autres
Publié: (2024)

Constructing Multilingual Visual-Text Datasets Revealing Visual Multilingual Ability of Vision Language Models
par: Atuhurra, Jesse, et autres
Publié: (2024)

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
par: Soldaini, Luca, et autres
Publié: (2024)

Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language Models
par: Balde, Gunjan, et autres
Publié: (2024)

Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints
par: Song, Ran, et autres
Publié: (2024)