ContPhy: Continuum Physical Concept Learning and Reasoning from Videos
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909271312039936 |
|---|---|
| author | Zheng, Zhicheng Yan, Xin Chen, Zhenfang Wang, Jingzhou Lim, Qin Zhi Eddie Tenenbaum, Joshua B. Gan, Chuang |
| author_facet | Zheng, Zhicheng Yan, Xin Chen, Zhenfang Wang, Jingzhou Lim, Qin Zhi Eddie Tenenbaum, Joshua B. Gan, Chuang |
| contents | We introduce the Continuum Physical Dataset (ContPhy), a novel benchmark for assessing machine physical commonsense. ContPhy complements existing physical reasoning benchmarks by encompassing the inference of diverse physical properties, such as mass and density, across various scenarios and predicting corresponding dynamics. We evaluated a range of AI models and found that they still struggle to achieve satisfactory performance on ContPhy, which shows that the current AI models still lack physical commonsense for the continuum, especially soft-bodies, and illustrates the value of the proposed dataset. We also introduce an oracle model (ContPRO) that marries the particle-based physical dynamic models with the recent large language models, which enjoy the advantages of both models, precise dynamic predictions, and interpretable reasoning. ContPhy aims to spur progress in perception and reasoning within diverse physical settings, narrowing the divide between human and machine intelligence in understanding the physical world. Project page: https://physical-reasoning-project.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_06119 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ContPhy: Continuum Physical Concept Learning and Reasoning from Videos Zheng, Zhicheng Yan, Xin Chen, Zhenfang Wang, Jingzhou Lim, Qin Zhi Eddie Tenenbaum, Joshua B. Gan, Chuang Computer Vision and Pattern Recognition We introduce the Continuum Physical Dataset (ContPhy), a novel benchmark for assessing machine physical commonsense. ContPhy complements existing physical reasoning benchmarks by encompassing the inference of diverse physical properties, such as mass and density, across various scenarios and predicting corresponding dynamics. We evaluated a range of AI models and found that they still struggle to achieve satisfactory performance on ContPhy, which shows that the current AI models still lack physical commonsense for the continuum, especially soft-bodies, and illustrates the value of the proposed dataset. We also introduce an oracle model (ContPRO) that marries the particle-based physical dynamic models with the recent large language models, which enjoy the advantages of both models, precise dynamic predictions, and interpretable reasoning. ContPhy aims to spur progress in perception and reasoning within diverse physical settings, narrowing the divide between human and machine intelligence in understanding the physical world. Project page: https://physical-reasoning-project.github.io |
| title | ContPhy: Continuum Physical Concept Learning and Reasoning from Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.06119 |