Self-Directed Synthetic Dialogues and Revisions Technical Report
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914888209661952 |
|---|---|
| author | Lambert, Nathan Schoelkopf, Hailey Gokaslan, Aaron Soldaini, Luca Pyatkin, Valentina Castricato, Louis |
| author_facet | Lambert, Nathan Schoelkopf, Hailey Gokaslan, Aaron Soldaini, Luca Pyatkin, Valentina Castricato, Louis |
| contents | Synthetic data has become an important tool in the fine-tuning of language models to follow instructions and solve complex problems. Nevertheless, the majority of open data to date is often lacking multi-turn data and collected on closed models, limiting progress on advancing open fine-tuning methods. We introduce Self Directed Synthetic Dialogues (SDSD), an experimental dataset consisting of guided conversations of language models talking to themselves. The dataset consists of multi-turn conversations generated with DBRX, Llama 2 70B, and Mistral Large, all instructed to follow a conversation plan generated prior to the conversation. We also explore including principles from Constitutional AI and other related works to create synthetic preference data via revisions to the final conversation turn. We hope this work encourages further exploration in multi-turn data and the use of open models for expanding the impact of synthetic data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_18421 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Self-Directed Synthetic Dialogues and Revisions Technical Report Lambert, Nathan Schoelkopf, Hailey Gokaslan, Aaron Soldaini, Luca Pyatkin, Valentina Castricato, Louis Computation and Language Machine Learning Synthetic data has become an important tool in the fine-tuning of language models to follow instructions and solve complex problems. Nevertheless, the majority of open data to date is often lacking multi-turn data and collected on closed models, limiting progress on advancing open fine-tuning methods. We introduce Self Directed Synthetic Dialogues (SDSD), an experimental dataset consisting of guided conversations of language models talking to themselves. The dataset consists of multi-turn conversations generated with DBRX, Llama 2 70B, and Mistral Large, all instructed to follow a conversation plan generated prior to the conversation. We also explore including principles from Constitutional AI and other related works to create synthetic preference data via revisions to the final conversation turn. We hope this work encourages further exploration in multi-turn data and the use of open models for expanding the impact of synthetic data. |
| title | Self-Directed Synthetic Dialogues and Revisions Technical Report |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2407.18421 |