Cover Image

Silent Tokens, Loud Effects: Padding in LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Himelstein, Rom, LeVi, Amit, Belinkov, Yonatan, Mendelson, Avi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Silenced Biases: The Dark Side LLMs Learned to Refuse
by: Himelstein, Rom, et al.
Published: (2025)

You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
by: LeVi, Amit, et al.
Published: (2025)

Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models
by: Rahimi, Eliron, et al.
Published: (2026)

Jailbreak Attack Initializations as Extractors of Compliance Directions
by: Levi, Amit, et al.
Published: (2025)

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
by: Itzhak, Itay, et al.
Published: (2025)

From Feelings to Metrics: Understanding and Formalizing How Users Vibe-Test LLMs
by: Itzhak, Itay, et al.
Published: (2026)

Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Information
by: Iskander, Shadi, et al.
Published: (2024)

Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models
by: Toker, Michael, et al.
Published: (2025)

SAEs Are Good for Steering -- If You Select the Right Features
by: Arad, Dana, et al.
Published: (2025)

Fast Forwarding Low-Rank Training
by: Rahamim, Adir, et al.
Published: (2024)

Training a Bilingual Language Model by Mapping Tokens onto a Shared Character Space
by: Rom, Aviad, et al.
Published: (2024)

Silent Signals, Loud Impact: LLMs for Word-Sense Disambiguation of Coded Dog Whistles
by: Kruk, Julia, et al.
Published: (2024)

Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms
by: Hanna, Michael, et al.
Published: (2024)

REMIND: Input Loss Landscapes Reveal Residual Memorization in Post-Unlearning LLMs
by: Cohen, Liran, et al.
Published: (2025)

Hysteresis Activation Function for Efficient Inference
by: Kimhi, Moshe, et al.
Published: (2024)

Leveraging NTPs for Efficient Hallucination Detection in VLMs
by: Azachi, Ofir, et al.
Published: (2025)

Backward Lens: Projecting Language Model Gradients into the Vocabulary Space
by: Katz, Shahar, et al.
Published: (2024)

Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software
by: Kordonsky, Tomer, et al.
Published: (2026)

Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking
by: Prakash, Nikhil, et al.
Published: (2024)

Tools Fail: Detecting Silent Errors in Faulty Tools
by: Sun, Jimin, et al.
Published: (2024)

Representing LLMs in Prompt Semantic Task Space
by: Kashani, Idan, et al.
Published: (2025)

Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
by: Marks, Samuel, et al.
Published: (2024)

Position-aware Automatic Circuit Discovery
by: Haklay, Tal, et al.
Published: (2025)

Confidence Regulation Neurons in Language Models
by: Stolfo, Alessandro, et al.
Published: (2024)

Structured RAG for Answering Aggregative Questions
by: Koshorek, Omri, et al.
Published: (2025)

Toward a Theory of Tokenization in LLMs
by: Rajaraman, Nived, et al.
Published: (2024)

Beyond Early-Token Bias: Model-Specific and Language-Specific Position Effects in Multilingual LLMs
by: Menschikov, Mikhail, et al.
Published: (2025)

TOFU: A Task of Fictitious Unlearning for LLMs
by: Maini, Pratyush, et al.
Published: (2024)

Persuasion Tokens for Editing Factual Knowledge in LLMs
by: Youssef, Paul, et al.
Published: (2026)

ContraSim -- Analyzing Neural Representations Based on Contrastive Learning
by: Rahamim, Adir, et al.
Published: (2023)

Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs
by: Singh, Aaditya K., et al.
Published: (2024)

Improving Self Consistency in LLMs through Probabilistic Tokenization
by: Sathe, Ashutosh, et al.
Published: (2024)

Sparse patches adversarial attacks via extrapolating point-wise information
by: Nemcovsky, Yaniv, et al.
Published: (2024)

Language Models Need Inductive Biases to Count Inductively
by: Chang, Yingshan, et al.
Published: (2024)

Trained on Tokens, Calibrated on Concepts: The Emergence of Semantic Calibration in LLMs
by: Nakkiran, Preetum, et al.
Published: (2025)

Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance
by: Goldman, Omer, et al.
Published: (2024)

Random Initialization of Gated Sparse Adapters
by: Retault, Vi, et al.
Published: (2025)

The Impossibility of Fair LLMs
by: Anthis, Jacy, et al.
Published: (2024)

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
by: Ouyang, Xu, et al.
Published: (2024)

Exact Expressive Power of Transformers with Padding
by: Merrill, William, et al.
Published: (2025)