Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.15052 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912696657510400 |
|---|---|
| author | Wu, Zihui Gao, Haichang Wang, Ping Zhang, Shudong Liu, Zhaoxiang Lian, Shiguo |
| author_facet | Wu, Zihui Gao, Haichang Wang, Ping Zhang, Shudong Liu, Zhaoxiang Lian, Shiguo |
| contents | Glitch tokens, inputs that trigger unpredictable or anomalous behavior in Large Language Models (LLMs), pose significant challenges to model reliability and safety. Existing detection methods primarily rely on heuristic embedding patterns or statistical anomalies within internal representations, limiting their generalizability across different model architectures and potentially missing anomalies that deviate from observed patterns. We introduce GlitchMiner, an behavior-driven framework designed to identify glitch tokens by maximizing predictive entropy. Leveraging a gradient-guided local search strategy, GlitchMiner efficiently explores the discrete token space without relying on model-specific heuristics or large-batch sampling. Extensive experiments across ten LLMs from five major model families demonstrate that GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency, providing a generalizable and scalable solution for effective glitch token discovery. Code is available at [https://github.com/wooozihu/GlitchMiner] |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_15052 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimization Wu, Zihui Gao, Haichang Wang, Ping Zhang, Shudong Liu, Zhaoxiang Lian, Shiguo Artificial Intelligence Glitch tokens, inputs that trigger unpredictable or anomalous behavior in Large Language Models (LLMs), pose significant challenges to model reliability and safety. Existing detection methods primarily rely on heuristic embedding patterns or statistical anomalies within internal representations, limiting their generalizability across different model architectures and potentially missing anomalies that deviate from observed patterns. We introduce GlitchMiner, an behavior-driven framework designed to identify glitch tokens by maximizing predictive entropy. Leveraging a gradient-guided local search strategy, GlitchMiner efficiently explores the discrete token space without relying on model-specific heuristics or large-batch sampling. Extensive experiments across ten LLMs from five major model families demonstrate that GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency, providing a generalizable and scalable solution for effective glitch token discovery. Code is available at [https://github.com/wooozihu/GlitchMiner] |
| title | GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimization |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2410.15052 |