Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911292356296704 |
|---|---|
| author | Dong, Jiancheng Jia, Pengyue Peng, Jingyu Wang, Maolin Wang, Yuhao Su, Lixin Sun, Xin Wang, Shuaiqiang Yin, Dawei Zhao, Xiangyu |
| author_facet | Dong, Jiancheng Jia, Pengyue Peng, Jingyu Wang, Maolin Wang, Yuhao Su, Lixin Sun, Xin Wang, Shuaiqiang Yin, Dawei Zhao, Xiangyu |
| contents | Carefully engineered system prompts play a critical role in guiding the behavior of LLM agents, but their considerable length introduces significant drawbacks, including increased inference latency, higher computational cost, and reduced effective context length. This raises the question of whether such lengthy prompts can be replaced by a drastically reduced number of tokens while preserving their behavioral effect on downstream tasks. To enable this, we propose a lightweight three-stage training framework that learns a single prompt-specific Behavior-Equivalent token ([BE]). The framework first trains [BE] to encode the natural-language content of the original system prompt via reconstruction, and then distills the prompt 's downstream behavior into this single token. Importantly, our method requires no access to model internals, no auxiliary compression models, and no labeled responses. Empirical evaluations on three datasets show that a single [BE] token achieves up to a 3000x reduction in prompt length, while retaining about 98% of the downstream performance of the original system prompts. This substantially reduces inference cost and leaves almost the entire context window available for user inputs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_23271 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs Dong, Jiancheng Jia, Pengyue Peng, Jingyu Wang, Maolin Wang, Yuhao Su, Lixin Sun, Xin Wang, Shuaiqiang Yin, Dawei Zhao, Xiangyu Computation and Language Carefully engineered system prompts play a critical role in guiding the behavior of LLM agents, but their considerable length introduces significant drawbacks, including increased inference latency, higher computational cost, and reduced effective context length. This raises the question of whether such lengthy prompts can be replaced by a drastically reduced number of tokens while preserving their behavioral effect on downstream tasks. To enable this, we propose a lightweight three-stage training framework that learns a single prompt-specific Behavior-Equivalent token ([BE]). The framework first trains [BE] to encode the natural-language content of the original system prompt via reconstruction, and then distills the prompt 's downstream behavior into this single token. Importantly, our method requires no access to model internals, no auxiliary compression models, and no labeled responses. Empirical evaluations on three datasets show that a single [BE] token achieves up to a 3000x reduction in prompt length, while retaining about 98% of the downstream performance of the original system prompts. This substantially reduces inference cost and leaves almost the entire context window available for user inputs. |
| title | Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.23271 |