A Hard Nut to Crack: Idiom Detection with Conversational Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911879295664128 |
|---|---|
| author | Fornaciari, Francesca De Luca Altuna, Begoña Gonzalez-Dios, Itziar Melero, Maite |
| author_facet | Fornaciari, Francesca De Luca Altuna, Begoña Gonzalez-Dios, Itziar Melero, Maite |
| contents | In this work, we explore idiomatic language processing with Large Language Models (LLMs). We introduce the Idiomatic language Test Suite IdioTS, a new dataset of difficult examples specifically designed by language experts to assess the capabilities of LLMs to process figurative language at sentence level. We propose a comprehensive evaluation methodology based on an idiom detection task, where LLMs are prompted with detecting an idiomatic expression in a given English sentence. We present a thorough automatic and manual evaluation of the results and an extensive error analysis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_10579 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Hard Nut to Crack: Idiom Detection with Conversational Large Language Models Fornaciari, Francesca De Luca Altuna, Begoña Gonzalez-Dios, Itziar Melero, Maite Computation and Language In this work, we explore idiomatic language processing with Large Language Models (LLMs). We introduce the Idiomatic language Test Suite IdioTS, a new dataset of difficult examples specifically designed by language experts to assess the capabilities of LLMs to process figurative language at sentence level. We propose a comprehensive evaluation methodology based on an idiom detection task, where LLMs are prompted with detecting an idiomatic expression in a given English sentence. We present a thorough automatic and manual evaluation of the results and an extensive error analysis. |
| title | A Hard Nut to Crack: Idiom Detection with Conversational Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2405.10579 |