NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910203240251392 |
|---|---|
| author | Lan, Jian Liu, Zhicheng Wang, Xinpeng Zhou, Yuhao Chen, Haokun Lv, Jiancheng Plank, Barbara Seidl, Thomas |
| author_facet | Lan, Jian Liu, Zhicheng Wang, Xinpeng Zhou, Yuhao Chen, Haokun Lv, Jiancheng Plank, Barbara Seidl, Thomas |
| contents | The ability to derive precise spatial and physical insights is a cornerstone of vision-language models (VLMs), yet their poor performances in related spatial intelligence tasks such as physical reasoning remain a fundamental barrier. The community critically lacks a scientific analysis revealing whether VLMs faithfully reach answers or plausibly make guesses. This work aims to provide a fundamental understanding of how VLMs perceive the physical world, and utilize physical laws, while assessing the reliability of model confidence. We propose NICE and FACT, a dual-diagnostic paradigm that explicitly decomposes quantitative reasoning for kinematic physics: FACT diagnoses visual fidelity, physical law comprehension, and temporal grounding. NICE studies our novel neighborhood-informed calibration method and novel metrics to evaluate and calibrate confidence reliability. Evaluated across 6 latest state-of-the-art VLMs, we uncover that models fail to identify visual preconditions or utilize necessary physical laws to reach answers. This work highlights and establishes a standardized diagnostic paradigm to guide the development of faithful, physically-grounded VLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_08452 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics Lan, Jian Liu, Zhicheng Wang, Xinpeng Zhou, Yuhao Chen, Haokun Lv, Jiancheng Plank, Barbara Seidl, Thomas Computer Vision and Pattern Recognition The ability to derive precise spatial and physical insights is a cornerstone of vision-language models (VLMs), yet their poor performances in related spatial intelligence tasks such as physical reasoning remain a fundamental barrier. The community critically lacks a scientific analysis revealing whether VLMs faithfully reach answers or plausibly make guesses. This work aims to provide a fundamental understanding of how VLMs perceive the physical world, and utilize physical laws, while assessing the reliability of model confidence. We propose NICE and FACT, a dual-diagnostic paradigm that explicitly decomposes quantitative reasoning for kinematic physics: FACT diagnoses visual fidelity, physical law comprehension, and temporal grounding. NICE studies our novel neighborhood-informed calibration method and novel metrics to evaluate and calibrate confidence reliability. Evaluated across 6 latest state-of-the-art VLMs, we uncover that models fail to identify visual preconditions or utilize necessary physical laws to reach answers. This work highlights and establishes a standardized diagnostic paradigm to guide the development of faithful, physically-grounded VLMs. |
| title | NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.08452 |