TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913577555722240 |
|---|---|
| author | Moskvoretskii, Viktor Neminova, Ekaterina Lobanova, Alina Panchenko, Alexander Nikishina, Irina |
| author_facet | Moskvoretskii, Viktor Neminova, Ekaterina Lobanova, Alina Panchenko, Alexander Nikishina, Irina |
| contents | In this paper, we explore the capabilities of LLMs in capturing lexical-semantic knowledge from WordNet on the example of the LLaMA-2-7b model and test it on multiple lexical semantic tasks. As the outcome of our experiments, we present TaxoLLaMA, the everything-in-one model, lightweight due to 4-bit quantization and LoRA. It achieves 11 SotA results, 4 top-2 results out of 16 tasks for the Taxonomy Enrichment, Hypernym Discovery, Taxonomy Construction, and Lexical Entailment tasks. Moreover, it demonstrates very strong zero-shot performance on Lexical Entailment and Taxonomy Construction with no fine-tuning. We also explore its hidden multilingual and domain adaptation capabilities with a little tuning or few-shot learning. All datasets, code, and model are available online at https://github.com/VityaVitalich/TaxoLLaMA |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_09207 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks Moskvoretskii, Viktor Neminova, Ekaterina Lobanova, Alina Panchenko, Alexander Nikishina, Irina Computation and Language In this paper, we explore the capabilities of LLMs in capturing lexical-semantic knowledge from WordNet on the example of the LLaMA-2-7b model and test it on multiple lexical semantic tasks. As the outcome of our experiments, we present TaxoLLaMA, the everything-in-one model, lightweight due to 4-bit quantization and LoRA. It achieves 11 SotA results, 4 top-2 results out of 16 tasks for the Taxonomy Enrichment, Hypernym Discovery, Taxonomy Construction, and Lexical Entailment tasks. Moreover, it demonstrates very strong zero-shot performance on Lexical Entailment and Taxonomy Construction with no fine-tuning. We also explore its hidden multilingual and domain adaptation capabilities with a little tuning or few-shot learning. All datasets, code, and model are available online at https://github.com/VityaVitalich/TaxoLLaMA |
| title | TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.09207 |