Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909630833098752 |
|---|---|
| author | Wagner, Eitan Alon, Nitay Barnby, Joseph M. Abend, Omri |
| author_facet | Wagner, Eitan Alon, Nitay Barnby, Joseph M. Abend, Omri |
| contents | Theory of Mind (ToM) capabilities in LLMs have recently become a central object of investigation. Cognitive science distinguishes between two steps required for ToM tasks: 1) determine whether to invoke ToM, which includes the appropriate Depth of Mentalizing (DoM), or level of recursion required to complete a task; and 2) applying the correct inference given the DoM. In this position paper, we first identify several lines of work in different communities in AI, including LLM benchmarking, ToM add-ons, ToM probing, and formal models for ToM. We argue that recent work in AI tends to focus exclusively on the second step which are typically framed as static logic problems. We conclude with suggestions for improved evaluation of ToM capabilities inspired by dynamic environments used in cognitive tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_13631 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning Wagner, Eitan Alon, Nitay Barnby, Joseph M. Abend, Omri Artificial Intelligence Computation and Language Theory of Mind (ToM) capabilities in LLMs have recently become a central object of investigation. Cognitive science distinguishes between two steps required for ToM tasks: 1) determine whether to invoke ToM, which includes the appropriate Depth of Mentalizing (DoM), or level of recursion required to complete a task; and 2) applying the correct inference given the DoM. In this position paper, we first identify several lines of work in different communities in AI, including LLM benchmarking, ToM add-ons, ToM probing, and formal models for ToM. We argue that recent work in AI tends to focus exclusively on the second step which are typically framed as static logic problems. We conclude with suggestions for improved evaluation of ToM capabilities inspired by dynamic environments used in cognitive tasks. |
| title | Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2412.13631 |