Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge
Fuente:
arXiv
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909424641114112 |
|---|---|
| author | Zhang, Haotong |
| author_facet | Zhang, Haotong |
| contents | We carry out a series of experiments to test large language models' multi-hop reasoning ability from three aspects: selecting and combining external knowledge, dealing with non-sequential reasoning tasks and generalising to data samples with larger numbers of hops. We test the GPT-3.5 model on four reasoning benchmarks with Chain-of-Thought prompting (and its variations). Our results reveal that despite the amazing performance achieved by large language models on various reasoning tasks, models still suffer from severe drawbacks which shows a large gap with humans. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_08317 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Zhang, Haotong Computation and Language We carry out a series of experiments to test large language models' multi-hop reasoning ability from three aspects: selecting and combining external knowledge, dealing with non-sequential reasoning tasks and generalising to data samples with larger numbers of hops. We test the GPT-3.5 model on four reasoning benchmarks with Chain-of-Thought prompting (and its variations). Our results reveal that despite the amazing performance achieved by large language models on various reasoning tasks, models still suffer from severe drawbacks which shows a large gap with humans. |
| title | Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.08317 |