TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918147398828032 |
|---|---|
| author | Liu, Yiwei Pretty, Emma Jane Huang, Jiahao Sugawara, Saku |
| author_facet | Liu, Yiwei Pretty, Emma Jane Huang, Jiahao Sugawara, Saku |
| contents | While recent studies explore Large Language Models' (LLMs) performance on Theory of Mind (ToM) reasoning tasks, research on ToM abilities that require more nuanced social context is limited, such as white lies. We introduce TactfulToM, a novel English benchmark designed to evaluate LLMs' ability to understand white lies within real-life conversations and reason about prosocial motivations behind them, particularly when they are used to spare others' feelings and maintain social harmony. Our benchmark is generated through a multi-stage human-in-the-loop pipeline where LLMs expand manually designed seed stories into conversations to maintain the information asymmetry between participants necessary for authentic white lies. We show that TactfulToM is challenging for state-of-the-art models, which perform substantially below humans, revealing shortcomings in their ability to fully comprehend the ToM reasoning that enables true understanding of white lies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17054 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies? Liu, Yiwei Pretty, Emma Jane Huang, Jiahao Sugawara, Saku Computation and Language Artificial Intelligence While recent studies explore Large Language Models' (LLMs) performance on Theory of Mind (ToM) reasoning tasks, research on ToM abilities that require more nuanced social context is limited, such as white lies. We introduce TactfulToM, a novel English benchmark designed to evaluate LLMs' ability to understand white lies within real-life conversations and reason about prosocial motivations behind them, particularly when they are used to spare others' feelings and maintain social harmony. Our benchmark is generated through a multi-stage human-in-the-loop pipeline where LLMs expand manually designed seed stories into conversations to maintain the information asymmetry between participants necessary for authentic white lies. We show that TactfulToM is challenging for state-of-the-art models, which perform substantially below humans, revealing shortcomings in their ability to fully comprehend the ToM reasoning that enables true understanding of white lies. |
| title | TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies? |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.17054 |