Are LLMs Good Cryptic Crossword Solvers?
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916563163021312 |
|---|---|
| author | Sadallah, Abdelrahman Kotova, Daria Kochmar, Ekaterina |
| author_facet | Sadallah, Abdelrahman Kotova, Daria Kochmar, Ekaterina |
| contents | Cryptic crosswords are puzzles that rely not only on general knowledge but also on the solver's ability to manipulate language on different levels and deal with various types of wordplay. Previous research suggests that solving such puzzles is a challenge even for modern NLP models. However, the abilities of large language models (LLMs) have not yet been tested on this task. In this paper, we establish the benchmark results for three popular LLMs -- LLaMA2, Mistral, and ChatGPT -- showing that their performance on this task is still far from that of humans. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12094 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Are LLMs Good Cryptic Crossword Solvers? Sadallah, Abdelrahman Kotova, Daria Kochmar, Ekaterina Artificial Intelligence Computation and Language Machine Learning Cryptic crosswords are puzzles that rely not only on general knowledge but also on the solver's ability to manipulate language on different levels and deal with various types of wordplay. Previous research suggests that solving such puzzles is a challenge even for modern NLP models. However, the abilities of large language models (LLMs) have not yet been tested on this task. In this paper, we establish the benchmark results for three popular LLMs -- LLaMA2, Mistral, and ChatGPT -- showing that their performance on this task is still far from that of humans. |
| title | Are LLMs Good Cryptic Crossword Solvers? |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2403.12094 |