Cover Image

Sticking to the Mean: Detecting Sticky Tokens in Text Embedding Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Kexin, Wang, Dongxia, Liu, Yi, Zhang, Haonan, Wang, Wenhai
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks
by: Chen, Kexin, et al.
Published: (2024)

ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal
by: Zhang, Haonan, et al.
Published: (2025)

LLM-VA: Resolving the Jailbreak-Overrefusal Trade-off via Vector Alignment
by: Zhang, Haonan, et al.
Published: (2026)

SwiftEmbed: Ultra-Fast Text Embeddings via Static Token Lookup for Real-Time Applications
by: Lansiaux, Edouard, et al.
Published: (2025)

Adversarial Attacks on AI-Generated Text Detection Models: A Token Probability-Based Approach Using Embeddings
by: Kadhim, Ahmed K., et al.
Published: (2025)

TAD-Bench: A Comprehensive Benchmark for Embedding-Based Text Anomaly Detection
by: Cao, Yang, et al.
Published: (2025)

Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models
by: Deng, Ningyuan, et al.
Published: (2025)

Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions
by: Li, Zhiwen, et al.
Published: (2025)

TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models
by: Yang, Zhen, et al.
Published: (2023)

Bias in Text Embedding Models
by: Rakivnenko, Vasyl, et al.
Published: (2024)

Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddings
by: Li, Shiyu, et al.
Published: (2025)

PixelPrune: Pixel-Level Adaptive Visual Token Reduction via Predictive Coding
by: Wang, Nan, et al.
Published: (2026)

Compressing Sequences in the Latent Embedding Space: $K$-Token Merging for Large Language Models
by: Xu, Zihao, et al.
Published: (2026)

Probing Language Models for Pre-training Data Detection
by: Liu, Zhenhua, et al.
Published: (2024)

GlitchProber: Advancing Effective Detection and Mitigation of Glitch Tokens in Large Language Models
by: Zhang, Zhibo, et al.
Published: (2024)

Epistemological Bias As a Means for the Automated Detection of Injustices in Text
by: Andrews, Kenya, et al.
Published: (2024)

Full-ECE: A Metric For Token-level Calibration on Large Language Models
by: Liu, Han, et al.
Published: (2024)

Reverse Probing: Supervised Token-level Uncertainty Quantification for Large Language Models in Clinical Text
by: Xiao, Bushi, et al.
Published: (2026)

Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token
by: Lin, Ailiang, et al.
Published: (2025)

Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Models
by: Wu, Jialiang, et al.
Published: (2025)

Training Text-to-Molecule Models with Context-Aware Tokenization
by: Kim, Seojin, et al.
Published: (2025)

Text Generation Beyond Discrete Token Sampling
by: Zhuang, Yufan, et al.
Published: (2025)

Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
by: Cho, Gyeongje, et al.
Published: (2025)

Gecko: Versatile Text Embeddings Distilled from Large Language Models
by: Lee, Jinhyuk, et al.
Published: (2024)

EmbeddingGemma: Powerful and Lightweight Text Representations
by: Vera, Henrique Schechter, et al.
Published: (2025)

Multiscale Positive-Unlabeled Detection of AI-Generated Texts
by: Tian, Yuchuan, et al.
Published: (2023)

Hey, That's My Data! Token-Only Dataset Inference in Large Language Models
by: Xiong, Chen, et al.
Published: (2025)

Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning
by: Sun, Yiqun, et al.
Published: (2025)

UniMoT: Unified Molecule-Text Language Model with Discrete Token Representation
by: Guo, Shuhan, et al.
Published: (2024)

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models
by: Wang, Zihan, et al.
Published: (2024)

Carrot and Stick: Inducing Self-Motivation with Positive & Negative Feedback
by: Sohn, Jimin, et al.
Published: (2024)

Linearly-Interpretable Concept Embedding Models for Text Analysis
by: De Santis, Francesco, et al.
Published: (2024)

TPA: Next Token Probability Attribution for Detecting Hallucinations in RAG
by: Lu, Pengqian, et al.
Published: (2025)

MuLan: Adapting Multilingual Diffusion Models for Hundreds of Languages with Negligible Cost
by: Xing, Sen, et al.
Published: (2024)

A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding
by: Lu, Jinghui, et al.
Published: (2024)

Reinforcement Learning with Token-level Feedback for Controllable Text Generation
by: Li, Wendi, et al.
Published: (2024)

ToBlend: Token-Level Blending With an Ensemble of LLMs to Attack AI-Generated Text Detection
by: Huang, Fan, et al.
Published: (2024)

Large Language Models can be Guided to Evade AI-Generated Text Detection
by: Lu, Ning, et al.
Published: (2023)

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training
by: Qing, Liu, et al.
Published: (2026)

TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
by: Zeng, Runjia, et al.
Published: (2026)