Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918312800157696 |
|---|---|
| author | Huo, Yu Zhang, Siyu Zeng, Kun Liu, Haoyue Lee, Owen Chen, Junlin Lu, Yuquan Guo, Yifu Liang, Yaodong Tang, Xiaoying |
| author_facet | Huo, Yu Zhang, Siyu Zeng, Kun Liu, Haoyue Lee, Owen Chen, Junlin Lu, Yuquan Guo, Yifu Liang, Yaodong Tang, Xiaoying |
| contents | Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints-notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose Shape-of-Thought (SoT), a visual CoT framework that enables progressive shape assembly via coherent 2D projections without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. To support this paradigm, we introduce SoT-26K, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and T2S-CompBench, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4% on component numeracy and 84.8% on structural topology, outperforming text-only baselines by around 20%. SoT establishes a new paradigm for transparent, process-supervised compositional generation. The code is available at https://anonymous.4open.science/r/16FE/. The SoT-26K dataset will be released upon acceptance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_21081 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought Huo, Yu Zhang, Siyu Zeng, Kun Liu, Haoyue Lee, Owen Chen, Junlin Lu, Yuquan Guo, Yifu Liang, Yaodong Tang, Xiaoying Computer Vision and Pattern Recognition Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints-notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose Shape-of-Thought (SoT), a visual CoT framework that enables progressive shape assembly via coherent 2D projections without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. To support this paradigm, we introduce SoT-26K, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and T2S-CompBench, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4% on component numeracy and 84.8% on structural topology, outperforming text-only baselines by around 20%. SoT establishes a new paradigm for transparent, process-supervised compositional generation. The code is available at https://anonymous.4open.science/r/16FE/. The SoT-26K dataset will be released upon acceptance. |
| title | Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.21081 |