Data Diversity Matters for Robust Instruction Tuning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915010970648576 |
|---|---|
| author | Bukharin, Alexander Li, Shiyang Wang, Zhengyang Yang, Jingfeng Yin, Bing Li, Xian Zhang, Chao Zhao, Tuo Jiang, Haoming |
| author_facet | Bukharin, Alexander Li, Shiyang Wang, Zhengyang Yang, Jingfeng Yin, Bing Li, Xian Zhang, Chao Zhao, Tuo Jiang, Haoming |
| contents | Recent works have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. However, creating such datasets is difficult and most works rely on manual curation or proprietary language models. Automatic data curation is difficult as it is still not clear how we can define diversity for instruction tuning, how diversity and quality depend on one other, and how we can optimize dataset quality and diversity. To resolve these issue, we propose a new algorithm, Quality-Diversity Instruction Tuning (QDIT). QDIT provides a simple method to simultaneously control dataset diversity and quality, allowing us to conduct an in-depth study on the effect of diversity and quality on instruction tuning performance. From this study we draw two key insights (1) there is a natural tradeoff between data diversity and quality and (2) increasing data diversity significantly improves the worst case instruction following performance, therefore improving robustness. We validate the performance of QDIT on several large scale instruction tuning datasets, where we find it can substantially improve worst and average case performance compared to quality-driven data selection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_14736 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Data Diversity Matters for Robust Instruction Tuning Bukharin, Alexander Li, Shiyang Wang, Zhengyang Yang, Jingfeng Yin, Bing Li, Xian Zhang, Chao Zhao, Tuo Jiang, Haoming Computation and Language Machine Learning Recent works have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. However, creating such datasets is difficult and most works rely on manual curation or proprietary language models. Automatic data curation is difficult as it is still not clear how we can define diversity for instruction tuning, how diversity and quality depend on one other, and how we can optimize dataset quality and diversity. To resolve these issue, we propose a new algorithm, Quality-Diversity Instruction Tuning (QDIT). QDIT provides a simple method to simultaneously control dataset diversity and quality, allowing us to conduct an in-depth study on the effect of diversity and quality on instruction tuning performance. From this study we draw two key insights (1) there is a natural tradeoff between data diversity and quality and (2) increasing data diversity significantly improves the worst case instruction following performance, therefore improving robustness. We validate the performance of QDIT on several large scale instruction tuning datasets, where we find it can substantially improve worst and average case performance compared to quality-driven data selection. |
| title | Data Diversity Matters for Robust Instruction Tuning |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2311.14736 |