Flaming-hot Initiation with Regular Execution Sampling for Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912231854178304 |
|---|---|
| author | Chen, Weizhe Zhang, Zhicheng Liu, Guanlin Zheng, Renjie Shi, Wenlei Dun, Chen Wu, Zheng Jin, Xing Yan, Lin |
| author_facet | Chen, Weizhe Zhang, Zhicheng Liu, Guanlin Zheng, Renjie Shi, Wenlei Dun, Chen Wu, Zheng Jin, Xing Yan, Lin |
| contents | Since the release of ChatGPT, large language models (LLMs) have demonstrated remarkable capabilities across various domains. A key challenge in developing these general capabilities is efficiently sourcing diverse, high-quality data. This becomes especially critical in reasoning-related tasks with sandbox checkers, such as math or code, where the goal is to generate correct solutions to specific problems with higher probability. In this work, we introduce Flaming-hot Initiation with Regular Execution (FIRE) sampling, a simple yet highly effective method to efficiently find good responses. Our empirical findings show that FIRE sampling enhances inference-time generation quality and also benefits training in the alignment stage. Furthermore, we explore how FIRE sampling improves performance by promoting diversity and analyze the impact of employing FIRE at different positions within a response. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_21236 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Flaming-hot Initiation with Regular Execution Sampling for Large Language Models Chen, Weizhe Zhang, Zhicheng Liu, Guanlin Zheng, Renjie Shi, Wenlei Dun, Chen Wu, Zheng Jin, Xing Yan, Lin Machine Learning Artificial Intelligence Computation and Language Since the release of ChatGPT, large language models (LLMs) have demonstrated remarkable capabilities across various domains. A key challenge in developing these general capabilities is efficiently sourcing diverse, high-quality data. This becomes especially critical in reasoning-related tasks with sandbox checkers, such as math or code, where the goal is to generate correct solutions to specific problems with higher probability. In this work, we introduce Flaming-hot Initiation with Regular Execution (FIRE) sampling, a simple yet highly effective method to efficiently find good responses. Our empirical findings show that FIRE sampling enhances inference-time generation quality and also benefits training in the alignment stage. Furthermore, we explore how FIRE sampling improves performance by promoting diversity and analyze the impact of employing FIRE at different positions within a response. |
| title | Flaming-hot Initiation with Regular Execution Sampling for Large Language Models |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2410.21236 |