Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908682014425088 |
|---|---|
| author | Cai, Xiaodong Lin, Hai Zhan, Shaoxiong Luo, Weiqi Kim, Hong-Gee Hao, Hongyan Yang, Yu Zheng, Hai-Tao |
| author_facet | Cai, Xiaodong Lin, Hai Zhan, Shaoxiong Luo, Weiqi Kim, Hong-Gee Hao, Hongyan Yang, Yu Zheng, Hai-Tao |
| contents | Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring extensive tuning and complicating deployment. We present Entropy Equilibrium Sampling (EES), an auxiliary hyperparameter-free approach inspired by information theory that can dynamically adjust candidate sets by balancing normalized entropy with probability mass. We evaluate EES on both reasoning and generation tasks across a range of model architectures. Our results show that EES consistently performs well across temperature settings, delivering competitive accuracy and coherence while maintaining diversity. By eliminating the need for hyperparameter tuning, EES greatly simplifies deployment while improving performance. Code is available at https://github.com/shuanncai/EES |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_00789 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation Cai, Xiaodong Lin, Hai Zhan, Shaoxiong Luo, Weiqi Kim, Hong-Gee Hao, Hongyan Yang, Yu Zheng, Hai-Tao Computation and Language Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring extensive tuning and complicating deployment. We present Entropy Equilibrium Sampling (EES), an auxiliary hyperparameter-free approach inspired by information theory that can dynamically adjust candidate sets by balancing normalized entropy with probability mass. We evaluate EES on both reasoning and generation tasks across a range of model architectures. Our results show that EES consistently performs well across temperature settings, delivering competitive accuracy and coherence while maintaining diversity. By eliminating the need for hyperparameter tuning, EES greatly simplifies deployment while improving performance. Code is available at https://github.com/shuanncai/EES |
| title | Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2512.00789 |