Small Language Models as Effective Guides for Large Language Models in Chinese Relation Extraction
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913620126859264 |
|---|---|
| author | Tang, Xuemei Wang, Jun |
| author_facet | Tang, Xuemei Wang, Jun |
| contents | Recently, large language models (LLMs) have been successful in relational extraction (RE) tasks, especially in the few-shot learning. An important problem in the field of RE is long-tailed data, while not much attention is paid to this problem using LLM approaches. Therefore, in this paper, we propose SLCoLM, a model collaboration framework, to mitigate the data long-tail problem. In our framework, we use the ``\textit{Training-Guide-Predict}'' strategy to combine the strengths of small pre-trained language models (SLMs) and LLMs, where a task-specific SLM framework acts as a guider, transfers task knowledge to the LLM and guides the LLM in performing RE tasks. Our experiments on an ancient Chinese RE dataset rich in relation types show that the approach facilitates RE of long-tail relation types. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14373 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Small Language Models as Effective Guides for Large Language Models in Chinese Relation Extraction Tang, Xuemei Wang, Jun Computation and Language Recently, large language models (LLMs) have been successful in relational extraction (RE) tasks, especially in the few-shot learning. An important problem in the field of RE is long-tailed data, while not much attention is paid to this problem using LLM approaches. Therefore, in this paper, we propose SLCoLM, a model collaboration framework, to mitigate the data long-tail problem. In our framework, we use the ``\textit{Training-Guide-Predict}'' strategy to combine the strengths of small pre-trained language models (SLMs) and LLMs, where a task-specific SLM framework acts as a guider, transfers task knowledge to the LLM and guides the LLM in performing RE tasks. Our experiments on an ancient Chinese RE dataset rich in relation types show that the approach facilitates RE of long-tail relation types. |
| title | Small Language Models as Effective Guides for Large Language Models in Chinese Relation Extraction |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.14373 |