Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.02318 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929234993217536 |
|---|---|
| author | Wang, Peiqi Shen, Yikang Guo, Zhen Stallone, Matthew Kim, Yoon Golland, Polina Panda, Rameswar |
| author_facet | Wang, Peiqi Shen, Yikang Guo, Zhen Stallone, Matthew Kim, Yoon Golland, Polina Panda, Rameswar |
| contents | We aim to select data subsets for the fine-tuning of large language models to more effectively follow instructions. Prior work has emphasized the importance of diversity in dataset curation but relied on heuristics such as the number of tasks. In this paper, we use determinantal point processes to capture the diversity and quality of instruction tuning datasets for subset selection. We propose to measure dataset diversity with log determinant distance that is the distance between the dataset of interest and a maximally diverse reference dataset. Our experiments demonstrate that the proposed diversity measure in the normalized weight gradient space is correlated with downstream instruction-following performance. Consequently, it can be used to inform when data selection is the most helpful and to analyze dataset curation strategies. We demonstrate the utility of our approach on various instruction tuning datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_02318 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Diversity Measurement and Subset Selection for Instruction Tuning Datasets Wang, Peiqi Shen, Yikang Guo, Zhen Stallone, Matthew Kim, Yoon Golland, Polina Panda, Rameswar Machine Learning Computation and Language We aim to select data subsets for the fine-tuning of large language models to more effectively follow instructions. Prior work has emphasized the importance of diversity in dataset curation but relied on heuristics such as the number of tasks. In this paper, we use determinantal point processes to capture the diversity and quality of instruction tuning datasets for subset selection. We propose to measure dataset diversity with log determinant distance that is the distance between the dataset of interest and a maximally diverse reference dataset. Our experiments demonstrate that the proposed diversity measure in the normalized weight gradient space is correlated with downstream instruction-following performance. Consequently, it can be used to inform when data selection is the most helpful and to analyze dataset curation strategies. We demonstrate the utility of our approach on various instruction tuning datasets. |
| title | Diversity Measurement and Subset Selection for Instruction Tuning Datasets |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2402.02318 |