EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909159085047808 |
|---|---|
| author | Zhang, Shimao Bao, Yu Huang, Shujian |
| author_facet | Zhang, Shimao Bao, Yu Huang, Shujian |
| contents | Recently, Large Language Models (LLMs) have demonstrated outstanding performance across a wide range of downstream language tasks. Temperature sampling is a commonly used decoding strategy for LLMs' generation process. However, a fixed temperature parameter is used in most cases, which may not always be an optimal choice for balancing generation quality and diversity. In this paper, we propose an effective Entropy-based Dynamic Temperature (EDT) Sampling method, to achieve a more balanced performance in terms of both generation quality and diversity by dynamically selecting the temperature parameter. Additionally, we also show model performance and comprehensive analyses for 4 different generation benchmarks. Our experiments show that EDT significantly outperforms the existing strategies across different tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14541 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling Zhang, Shimao Bao, Yu Huang, Shujian Computation and Language Recently, Large Language Models (LLMs) have demonstrated outstanding performance across a wide range of downstream language tasks. Temperature sampling is a commonly used decoding strategy for LLMs' generation process. However, a fixed temperature parameter is used in most cases, which may not always be an optimal choice for balancing generation quality and diversity. In this paper, we propose an effective Entropy-based Dynamic Temperature (EDT) Sampling method, to achieve a more balanced performance in terms of both generation quality and diversity by dynamically selecting the temperature parameter. Additionally, we also show model performance and comprehensive analyses for 4 different generation benchmarks. Our experiments show that EDT significantly outperforms the existing strategies across different tasks. |
| title | EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.14541 |