Cognitive Modeling with Scaffolded LLMs: A Case Study of Referential Expression Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929412688052224 |
|---|---|
| author | Tsvilodub, Polina Franke, Michael Carcassi, Fausto |
| author_facet | Tsvilodub, Polina Franke, Michael Carcassi, Fausto |
| contents | To what extent can LLMs be used as part of a cognitive model of language generation? In this paper, we approach this question by exploring a neuro-symbolic implementation of an algorithmic cognitive model of referential expression generation by Dale & Reiter (1995). The symbolic task analysis implements the generation as an iterative procedure that scaffolds symbolic and gpt-3.5-turbo-based modules. We compare this implementation to an ablated model and a one-shot LLM-only baseline on the A3DS dataset (Tsvilodub & Franke, 2023). We find that our hybrid approach is cognitively plausible and performs well in complex contexts, while allowing for more open-ended modeling of language generation in a larger domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_03805 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Cognitive Modeling with Scaffolded LLMs: A Case Study of Referential Expression Generation Tsvilodub, Polina Franke, Michael Carcassi, Fausto Computation and Language To what extent can LLMs be used as part of a cognitive model of language generation? In this paper, we approach this question by exploring a neuro-symbolic implementation of an algorithmic cognitive model of referential expression generation by Dale & Reiter (1995). The symbolic task analysis implements the generation as an iterative procedure that scaffolds symbolic and gpt-3.5-turbo-based modules. We compare this implementation to an ablated model and a one-shot LLM-only baseline on the A3DS dataset (Tsvilodub & Franke, 2023). We find that our hybrid approach is cognitively plausible and performs well in complex contexts, while allowing for more open-ended modeling of language generation in a larger domain. |
| title | Cognitive Modeling with Scaffolded LLMs: A Case Study of Referential Expression Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.03805 |