ShapeShift: Text-to-Mosaic Synthesis via Semantic Phase-Field Guidance
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908847922216960 |
|---|---|
| author | Misra, Vihaan Schaldenbrand, Peter Oh, Jean |
| author_facet | Misra, Vihaan Schaldenbrand, Peter Oh, Jean |
| contents | We present ShapeShift, a method for arranging rigid objects into configurations that visually convey semantic concepts specified by natural language. While pretrained diffusion models provide powerful semantic guidance, such as Score Distillation Sampling, enforcing physical validity poses a fundamental challenge. Naive overlap resolution disrupts semantic structure -- separating overlapping shapes along geometrically optimal directions (minimum translation vectors) often destroys the very arrangements that make concepts recognizable.
Our intuition is that diffusion model features encode not just what a concept looks like, but its geometric, directional structure -- how it is oriented and shaped -- which we leverage to make overlap resolution semantically aware. We introduce a deformable boundary represented as a phase field that expands anisotropically, guided by intermediate features from the diffusion model, creating space along semantically coherent directions. Experiments demonstrate that ShapeShift, by coupling semantic guidance and feasibility constraint resolution, produces arrangements achieving both semantic clarity and overlap-free validity, significantly outperforming baselines that treat these objectives independently. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_14720 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ShapeShift: Text-to-Mosaic Synthesis via Semantic Phase-Field Guidance Misra, Vihaan Schaldenbrand, Peter Oh, Jean Computer Vision and Pattern Recognition We present ShapeShift, a method for arranging rigid objects into configurations that visually convey semantic concepts specified by natural language. While pretrained diffusion models provide powerful semantic guidance, such as Score Distillation Sampling, enforcing physical validity poses a fundamental challenge. Naive overlap resolution disrupts semantic structure -- separating overlapping shapes along geometrically optimal directions (minimum translation vectors) often destroys the very arrangements that make concepts recognizable. Our intuition is that diffusion model features encode not just what a concept looks like, but its geometric, directional structure -- how it is oriented and shaped -- which we leverage to make overlap resolution semantically aware. We introduce a deformable boundary represented as a phase field that expands anisotropically, guided by intermediate features from the diffusion model, creating space along semantically coherent directions. Experiments demonstrate that ShapeShift, by coupling semantic guidance and feasibility constraint resolution, produces arrangements achieving both semantic clarity and overlap-free validity, significantly outperforming baselines that treat these objectives independently. |
| title | ShapeShift: Text-to-Mosaic Synthesis via Semantic Phase-Field Guidance |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.14720 |