Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Code
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909336288100352 |
|---|---|
| author | Chae, Hyungjoo Kwon, Taeyoon Moon, Seungjun Song, Yongho Kang, Dongjin Ong, Kai Tzu-iunn Kwak, Beong-woo Bae, Seonghyeon Hwang, Seung-won Yeo, Jinyoung |
| author_facet | Chae, Hyungjoo Kwon, Taeyoon Moon, Seungjun Song, Yongho Kang, Dongjin Ong, Kai Tzu-iunn Kwak, Beong-woo Bae, Seonghyeon Hwang, Seung-won Yeo, Jinyoung |
| contents | This paper presents Coffee-Gym, a comprehensive RL environment for training models that provide feedback on code editing. Coffee-Gym includes two major components: (1) Coffee, a dataset containing humans' code edit traces for coding questions and machine-written feedback for editing erroneous code; (2) CoffeeEval, a reward function that faithfully reflects the helpfulness of feedback by assessing the performance of the revised code in unit tests. With them, Coffee-Gym addresses the unavailability of high-quality datasets for training feedback models with RL, and provides more accurate rewards than the SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback models that outperform baselines in enhancing open-source code LLMs' code editing, making them comparable with closed-source LLMs. We make the dataset and the model checkpoint publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19715 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Code Chae, Hyungjoo Kwon, Taeyoon Moon, Seungjun Song, Yongho Kang, Dongjin Ong, Kai Tzu-iunn Kwak, Beong-woo Bae, Seonghyeon Hwang, Seung-won Yeo, Jinyoung Computation and Language This paper presents Coffee-Gym, a comprehensive RL environment for training models that provide feedback on code editing. Coffee-Gym includes two major components: (1) Coffee, a dataset containing humans' code edit traces for coding questions and machine-written feedback for editing erroneous code; (2) CoffeeEval, a reward function that faithfully reflects the helpfulness of feedback by assessing the performance of the revised code in unit tests. With them, Coffee-Gym addresses the unavailability of high-quality datasets for training feedback models with RL, and provides more accurate rewards than the SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback models that outperform baselines in enhancing open-source code LLMs' code editing, making them comparable with closed-source LLMs. We make the dataset and the model checkpoint publicly available. |
| title | Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Code |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.19715 |