CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915235498033152 |
|---|---|
| author | Chen, Cheng Wei, Jiacheng Chen, Tianrun Zhang, Chi Yang, Xiaofeng Zhang, Shangzhan Yang, Bingchen Foo, Chuan-Sheng Lin, Guosheng Huang, Qixing Liu, Fayao |
| author_facet | Chen, Cheng Wei, Jiacheng Chen, Tianrun Zhang, Chi Yang, Xiaofeng Zhang, Shangzhan Yang, Bingchen Foo, Chuan-Sheng Lin, Guosheng Huang, Qixing Liu, Fayao |
| contents | Creating CAD digital twins from the physical world is crucial for manufacturing, design, and simulation. However, current methods typically rely on costly 3D scanning with labor-intensive post-processing. To provide a user-friendly design process, we explore the problem of reverse engineering from unconstrained real-world CAD images that can be easily captured by users of all experiences. However, the scarcity of real-world CAD data poses challenges in directly training such models. To tackle these challenges, we propose CADCrafter, an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data while testing on real-world images. To bridge the significant representation disparity between images and parametric CAD models, we introduce a geometry encoder to accurately capture diverse geometric features. Moreover, the texture-invariant properties of the geometric features can also facilitate the generalization to real-world scenarios. Since compiling CAD parameter sequences into explicit CAD models is a non-differentiable process, the network training inherently lacks explicit geometric supervision. To impose geometric validity constraints, we employ direct preference optimization (DPO) to fine-tune our model with the automatic code checker feedback on CAD sequence quality. Furthermore, we collected a real-world dataset, comprised of multi-view images and corresponding CAD command sequence pairs, to evaluate our method. Experimental results demonstrate that our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04753 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images Chen, Cheng Wei, Jiacheng Chen, Tianrun Zhang, Chi Yang, Xiaofeng Zhang, Shangzhan Yang, Bingchen Foo, Chuan-Sheng Lin, Guosheng Huang, Qixing Liu, Fayao Computer Vision and Pattern Recognition Creating CAD digital twins from the physical world is crucial for manufacturing, design, and simulation. However, current methods typically rely on costly 3D scanning with labor-intensive post-processing. To provide a user-friendly design process, we explore the problem of reverse engineering from unconstrained real-world CAD images that can be easily captured by users of all experiences. However, the scarcity of real-world CAD data poses challenges in directly training such models. To tackle these challenges, we propose CADCrafter, an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data while testing on real-world images. To bridge the significant representation disparity between images and parametric CAD models, we introduce a geometry encoder to accurately capture diverse geometric features. Moreover, the texture-invariant properties of the geometric features can also facilitate the generalization to real-world scenarios. Since compiling CAD parameter sequences into explicit CAD models is a non-differentiable process, the network training inherently lacks explicit geometric supervision. To impose geometric validity constraints, we employ direct preference optimization (DPO) to fine-tune our model with the automatic code checker feedback on CAD sequence quality. Furthermore, we collected a real-world dataset, comprised of multi-view images and corresponding CAD command sequence pairs, to evaluate our method. Experimental results demonstrate that our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects. |
| title | CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.04753 |