ThoughtSource: A central hub for large language model reasoning data
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909118680268800 |
|---|---|
| author | Ott, Simon Hebenstreit, Konstantin Liévin, Valentin Hother, Christoffer Egeberg Moradi, Milad Mayrhauser, Maximilian Praas, Robert Winther, Ole Samwald, Matthias |
| author_facet | Ott, Simon Hebenstreit, Konstantin Liévin, Valentin Hother, Christoffer Egeberg Moradi, Milad Mayrhauser, Maximilian Praas, Robert Winther, Ole Samwald, Matthias |
| contents | Large language models (LLMs) such as GPT-4 have recently demonstrated impressive results across a wide range of tasks. LLMs are still limited, however, in that they frequently fail at complex reasoning, their reasoning processes are opaque, they are prone to 'hallucinate' facts, and there are concerns about their underlying biases. Letting models verbalize reasoning steps as natural language, a technique known as chain-of-thought prompting, has recently been proposed as a way to address some of these issues. Here we present ThoughtSource, a meta-dataset and software library for chain-of-thought (CoT) reasoning. The goal of ThoughtSource is to improve future artificial intelligence systems by facilitating qualitative understanding of CoTs, enabling empirical evaluations, and providing training data. This first release of ThoughtSource integrates seven scientific/medical, three general-domain and five math word question answering datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_11596 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | ThoughtSource: A central hub for large language model reasoning data Ott, Simon Hebenstreit, Konstantin Liévin, Valentin Hother, Christoffer Egeberg Moradi, Milad Mayrhauser, Maximilian Praas, Robert Winther, Ole Samwald, Matthias Computation and Language Artificial Intelligence Large language models (LLMs) such as GPT-4 have recently demonstrated impressive results across a wide range of tasks. LLMs are still limited, however, in that they frequently fail at complex reasoning, their reasoning processes are opaque, they are prone to 'hallucinate' facts, and there are concerns about their underlying biases. Letting models verbalize reasoning steps as natural language, a technique known as chain-of-thought prompting, has recently been proposed as a way to address some of these issues. Here we present ThoughtSource, a meta-dataset and software library for chain-of-thought (CoT) reasoning. The goal of ThoughtSource is to improve future artificial intelligence systems by facilitating qualitative understanding of CoTs, enabling empirical evaluations, and providing training data. This first release of ThoughtSource integrates seven scientific/medical, three general-domain and five math word question answering datasets. |
| title | ThoughtSource: A central hub for large language model reasoning data |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2301.11596 |