GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913255714193408 |
|---|---|
| author | Sainz, Oscar García-Ferrero, Iker Agerri, Rodrigo de Lacalle, Oier Lopez Rigau, German Agirre, Eneko |
| author_facet | Sainz, Oscar García-Ferrero, Iker Agerri, Rodrigo de Lacalle, Oier Lopez Rigau, German Agirre, Eneko |
| contents | Large Language Models (LLMs) combined with instruction tuning have made significant progress when generalizing to unseen tasks. However, they have been less successful in Information Extraction (IE), lagging behind task-specific models. Typically, IE tasks are characterized by complex annotation guidelines that describe the task and give examples to humans. Previous attempts to leverage such information have failed, even with the largest models, as they are not able to follow the guidelines out of the box. In this paper, we propose GoLLIE (Guideline-following Large Language Model for IE), a model able to improve zero-shot results on unseen IE tasks by virtue of being fine-tuned to comply with annotation guidelines. Comprehensive evaluation empirically demonstrates that GoLLIE is able to generalize to and follow unseen guidelines, outperforming previous attempts at zero-shot information extraction. The ablation study shows that detailed guidelines are key for good results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_03668 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction Sainz, Oscar García-Ferrero, Iker Agerri, Rodrigo de Lacalle, Oier Lopez Rigau, German Agirre, Eneko Computation and Language Large Language Models (LLMs) combined with instruction tuning have made significant progress when generalizing to unseen tasks. However, they have been less successful in Information Extraction (IE), lagging behind task-specific models. Typically, IE tasks are characterized by complex annotation guidelines that describe the task and give examples to humans. Previous attempts to leverage such information have failed, even with the largest models, as they are not able to follow the guidelines out of the box. In this paper, we propose GoLLIE (Guideline-following Large Language Model for IE), a model able to improve zero-shot results on unseen IE tasks by virtue of being fine-tuned to comply with annotation guidelines. Comprehensive evaluation empirically demonstrates that GoLLIE is able to generalize to and follow unseen guidelines, outperforming previous attempts at zero-shot information extraction. The ablation study shows that detailed guidelines are key for good results. |
| title | GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2310.03668 |