Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.07109 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910275120136192 |
|---|---|
| author | Luo, Dezhi Li, Yijiang Wang, Maijunxian Zhao, Tianwei Wang, Bingyang Wang, Siheng Feng, Pinyuan Rahmanzadehgervi, Pooyan Ma, Ziqiao Deng, Hokin |
| author_facet | Luo, Dezhi Li, Yijiang Wang, Maijunxian Zhao, Tianwei Wang, Bingyang Wang, Siheng Feng, Pinyuan Rahmanzadehgervi, Pooyan Ma, Ziqiao Deng, Hokin |
| contents | Understanding physical transformations is fundamental for reasoning in dynamic environments. While Vision Language Models (VLMs) show promise in embodied applications, whether they genuinely understand physical transformations remains unclear. We introduce ConservationBench evaluating conservation -- whether physical quantities remain invariant under transformations. Spanning four properties with paired conserving/non-conserving scenarios, we generate and evaluate 23,040 questions across 112 VLMs. Results reveal systematic failure: performance remains near chance with improvements on conservation tasks accompanied by drops on controls. Control experiments show strong textual priors favoring invariance, yet models perform worse with actual visual content when performance is balanced across conserving and non-conserving scenarios. Neither temporal resolution, prompting, nor curated sampling helps. These findings show that current VLMs fail to maintain transformation-invariant representations of physical properties across dynamic scenes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_07109 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Vision Language Models Cannot Reason About Physical Transformation Luo, Dezhi Li, Yijiang Wang, Maijunxian Zhao, Tianwei Wang, Bingyang Wang, Siheng Feng, Pinyuan Rahmanzadehgervi, Pooyan Ma, Ziqiao Deng, Hokin Artificial Intelligence Understanding physical transformations is fundamental for reasoning in dynamic environments. While Vision Language Models (VLMs) show promise in embodied applications, whether they genuinely understand physical transformations remains unclear. We introduce ConservationBench evaluating conservation -- whether physical quantities remain invariant under transformations. Spanning four properties with paired conserving/non-conserving scenarios, we generate and evaluate 23,040 questions across 112 VLMs. Results reveal systematic failure: performance remains near chance with improvements on conservation tasks accompanied by drops on controls. Control experiments show strong textual priors favoring invariance, yet models perform worse with actual visual content when performance is balanced across conserving and non-conserving scenarios. Neither temporal resolution, prompting, nor curated sampling helps. These findings show that current VLMs fail to maintain transformation-invariant representations of physical properties across dynamic scenes. |
| title | Vision Language Models Cannot Reason About Physical Transformation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2603.07109 |