Defining Knowledge: Bridging Epistemology and Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909334532784128 |
|---|---|
| author | Fierro, Constanza Dhar, Ruchira Stamatiou, Filippos Garneau, Nicolas Søgaard, Anders |
| author_facet | Fierro, Constanza Dhar, Ruchira Stamatiou, Filippos Garneau, Nicolas Søgaard, Anders |
| contents | Knowledge claims are abundant in the literature on large language models (LLMs); but can we say that GPT-4 truly "knows" the Earth is round? To address this question, we review standard definitions of knowledge in epistemology and we formalize interpretations applicable to LLMs. In doing so, we identify inconsistencies and gaps in how current NLP research conceptualizes knowledge with respect to epistemological frameworks. Additionally, we conduct a survey of 100 professional philosophers and computer scientists to compare their preferences in knowledge definitions and their views on whether LLMs can really be said to know. Finally, we suggest evaluation protocols for testing knowledge in accordance to the most relevant definitions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02499 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Defining Knowledge: Bridging Epistemology and Large Language Models Fierro, Constanza Dhar, Ruchira Stamatiou, Filippos Garneau, Nicolas Søgaard, Anders Computation and Language Knowledge claims are abundant in the literature on large language models (LLMs); but can we say that GPT-4 truly "knows" the Earth is round? To address this question, we review standard definitions of knowledge in epistemology and we formalize interpretations applicable to LLMs. In doing so, we identify inconsistencies and gaps in how current NLP research conceptualizes knowledge with respect to epistemological frameworks. Additionally, we conduct a survey of 100 professional philosophers and computer scientists to compare their preferences in knowledge definitions and their views on whether LLMs can really be said to know. Finally, we suggest evaluation protocols for testing knowledge in accordance to the most relevant definitions. |
| title | Defining Knowledge: Bridging Epistemology and Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.02499 |