The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912312230674432 |
|---|---|
| author | He, Bingxiang Ding, Ning Qian, Cheng Deng, Jia Cui, Ganqu Yuan, Lifan Hong, Haiwen Gao, Huan-ang Huang, Longtao Xue, Hui Chen, Huimin Liu, Zhiyuan Sun, Maosong |
| author_facet | He, Bingxiang Ding, Ning Qian, Cheng Deng, Jia Cui, Ganqu Yuan, Lifan Hong, Haiwen Gao, Huan-ang Huang, Longtao Xue, Hui Chen, Huimin Liu, Zhiyuan Sun, Maosong |
| contents | Understanding alignment techniques begins with comprehending zero-shot generalization brought by instruction tuning, but little of the mechanism has been understood. Existing work has largely been confined to the task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations. To bridge this gap, we investigate zero-shot generalization from the perspective of the data itself. We first demonstrate that zero-shot generalization happens very early during instruction tuning, with loss serving as a stable indicator. Next, we investigate training data arrangement through similarity and granularity perspectives, confirming that the timing of exposure to certain training examples may greatly facilitate generalization on unseen tasks. Finally, we propose a more grounded training data arrangement framework, Test-centric Multi-turn Arrangement, and show its effectiveness in promoting continual learning and further loss reduction. For the first time, we show that zero-shot generalization during instruction tuning is a form of similarity-based generalization between training and test data at the instance level. Our code is released at https://github.com/thunlp/Dynamics-of-Zero-Shot-Generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_11721 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning He, Bingxiang Ding, Ning Qian, Cheng Deng, Jia Cui, Ganqu Yuan, Lifan Hong, Haiwen Gao, Huan-ang Huang, Longtao Xue, Hui Chen, Huimin Liu, Zhiyuan Sun, Maosong Computation and Language Artificial Intelligence Machine Learning Understanding alignment techniques begins with comprehending zero-shot generalization brought by instruction tuning, but little of the mechanism has been understood. Existing work has largely been confined to the task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations. To bridge this gap, we investigate zero-shot generalization from the perspective of the data itself. We first demonstrate that zero-shot generalization happens very early during instruction tuning, with loss serving as a stable indicator. Next, we investigate training data arrangement through similarity and granularity perspectives, confirming that the timing of exposure to certain training examples may greatly facilitate generalization on unseen tasks. Finally, we propose a more grounded training data arrangement framework, Test-centric Multi-turn Arrangement, and show its effectiveness in promoting continual learning and further loss reduction. For the first time, we show that zero-shot generalization during instruction tuning is a form of similarity-based generalization between training and test data at the instance level. Our code is released at https://github.com/thunlp/Dynamics-of-Zero-Shot-Generalization. |
| title | The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2406.11721 |