CreativityBench: Evaluating Agent Creative Reasoning via Affordance-Based Tool Repurposing
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913093283479552 |
|---|---|
| author | Qian, Cheng Ha, Hyeonjeong Liu, Jiayu Kim, Jeonghwan Liu, Jiateng Li, Bingxuan Tiwari, Aditi Dalal, Dwip Wang, Zhenhailong Chen, Xiusi Namazifar, Mahdi Li, Yunzhu Ji, Heng |
| author_facet | Qian, Cheng Ha, Hyeonjeong Liu, Jiayu Kim, Jeonghwan Liu, Jiateng Li, Bingxuan Tiwari, Aditi Dalal, Dwip Wang, Zhenhailong Chen, Xiusi Namazifar, Mahdi Li, Yunzhu Ji, Heng |
| contents | Recent advances in large language models have led to strong performance on reasoning and environment-interaction tasks, yet their ability for creative problem-solving remains underexplored. We study this capability through the lens of creative tool use, where a model repurposes available objects by reasoning about their affordances and attributes rather than relying on canonical usage. As a first step, we introduce CreativityBench, a benchmark for evaluating affordance-based creativity in LLMs. To this end, we build a large-scale affordance knowledge base (KB) with 4K entities and 150K+ affordance annotations, explicitly linking objects, parts, attributes, and actionable uses. Building on this KB, we generate 14K grounded tasks that require identifying non-obvious yet physically plausible solutions under constraints. Evaluations across 10 state-of-the-art LLMs, including closed and open-source models, show that models can often select a plausible object, but fail to identify the correct parts, their affordances, and the underlying physical mechanism needed to solve the task, leading to a significant drop in performance. Furthermore, improvements from model scaling quickly saturate, strong general reasoning does not reliably translate to creative affordance discovery, and common inference-time strategies such as Chain-of-Thought yield limited gains. These results suggest that creative tool use remains a major challenge for current models, and that CreativityBench provides a useful testbed for studying this missing dimension of intelligence, with potential implications for planning and reasoning modules in future agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_02910 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CreativityBench: Evaluating Agent Creative Reasoning via Affordance-Based Tool Repurposing Qian, Cheng Ha, Hyeonjeong Liu, Jiayu Kim, Jeonghwan Liu, Jiateng Li, Bingxuan Tiwari, Aditi Dalal, Dwip Wang, Zhenhailong Chen, Xiusi Namazifar, Mahdi Li, Yunzhu Ji, Heng Artificial Intelligence Computation and Language Machine Learning Recent advances in large language models have led to strong performance on reasoning and environment-interaction tasks, yet their ability for creative problem-solving remains underexplored. We study this capability through the lens of creative tool use, where a model repurposes available objects by reasoning about their affordances and attributes rather than relying on canonical usage. As a first step, we introduce CreativityBench, a benchmark for evaluating affordance-based creativity in LLMs. To this end, we build a large-scale affordance knowledge base (KB) with 4K entities and 150K+ affordance annotations, explicitly linking objects, parts, attributes, and actionable uses. Building on this KB, we generate 14K grounded tasks that require identifying non-obvious yet physically plausible solutions under constraints. Evaluations across 10 state-of-the-art LLMs, including closed and open-source models, show that models can often select a plausible object, but fail to identify the correct parts, their affordances, and the underlying physical mechanism needed to solve the task, leading to a significant drop in performance. Furthermore, improvements from model scaling quickly saturate, strong general reasoning does not reliably translate to creative affordance discovery, and common inference-time strategies such as Chain-of-Thought yield limited gains. These results suggest that creative tool use remains a major challenge for current models, and that CreativityBench provides a useful testbed for studying this missing dimension of intelligence, with potential implications for planning and reasoning modules in future agents. |
| title | CreativityBench: Evaluating Agent Creative Reasoning via Affordance-Based Tool Repurposing |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2605.02910 |