TRACE: A Framework for Analyzing and Enhancing Stepwise Reasoning in Vision-Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917140604387328 |
|---|---|
| author | Imani, Shima Moon, Seungwhan Mathias, Lambert Zhang, Lu Damavandi, Babak |
| author_facet | Imani, Shima Moon, Seungwhan Mathias, Lambert Zhang, Lu Damavandi, Babak |
| contents | Reliable mathematical and scientific reasoning remains an open challenge for large vision-language models. Standard final-answer evaluation often masks reasoning errors, allowing silent failures to persist. To address this gap, we introduce TRACE, a framework for Transparent Reasoning And Consistency Evaluation that diagnoses reasoning trajectories rather than only end results. At its core, TRACE leverages Auxiliary Reasoning Sets, compact sub question answer pairs that decompose complex problems, evaluate intermediate steps through consistency-based metrics, and expose failures overlooked by standard evaluation. Our experiments show that consistency across ARS correlates with final-answer correctness and helps pinpoint the reasoning steps where failures arise, offering actionable signals for model improvement. Furthermore, TRACE defines confidence regions that distinguish reliable from unreliable reasoning paths, supporting effective filtering, debugging, and model refinement. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_05943 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TRACE: A Framework for Analyzing and Enhancing Stepwise Reasoning in Vision-Language Models Imani, Shima Moon, Seungwhan Mathias, Lambert Zhang, Lu Damavandi, Babak Artificial Intelligence Reliable mathematical and scientific reasoning remains an open challenge for large vision-language models. Standard final-answer evaluation often masks reasoning errors, allowing silent failures to persist. To address this gap, we introduce TRACE, a framework for Transparent Reasoning And Consistency Evaluation that diagnoses reasoning trajectories rather than only end results. At its core, TRACE leverages Auxiliary Reasoning Sets, compact sub question answer pairs that decompose complex problems, evaluate intermediate steps through consistency-based metrics, and expose failures overlooked by standard evaluation. Our experiments show that consistency across ARS correlates with final-answer correctness and helps pinpoint the reasoning steps where failures arise, offering actionable signals for model improvement. Furthermore, TRACE defines confidence regions that distinguish reliable from unreliable reasoning paths, supporting effective filtering, debugging, and model refinement. |
| title | TRACE: A Framework for Analyzing and Enhancing Stepwise Reasoning in Vision-Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.05943 |