cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912910076280832 |
|---|---|
| author | Kolodiazhnyi, Maksim Tarasov, Denis Zhemchuzhnikov, Dmitrii Nikulin, Alexander Zisman, Ilya Vorontsova, Anna Konushin, Anton Kurenkov, Vladislav Rukhovich, Danila |
| author_facet | Kolodiazhnyi, Maksim Tarasov, Denis Zhemchuzhnikov, Dmitrii Nikulin, Alexander Zisman, Ilya Vorontsova, Anna Konushin, Anton Kurenkov, Vladislav Rukhovich, Danila |
| contents | Computer-Aided Design (CAD) plays a central role in engineering and manufacturing, making it possible to create precise and editable 3D models. Using a variety of sensor or user-provided data as inputs for CAD reconstruction can democratize access to design applications. However, existing methods typically focus on a single input modality, such as point clouds, images, or text, which limits their generalizability and robustness. Leveraging recent advances in vision-language models (VLM), we propose a multi-modal CAD reconstruction model that simultaneously processes all three input modalities. Inspired by large language model (LLM) training paradigms, we adopt a two-stage pipeline: supervised fine-tuning (SFT) on large-scale procedurally generated data, followed by reinforcement learning (RL) fine-tuning using online feedback, obtained programatically. Furthermore, we are the first to explore RL fine-tuning of LLMs for CAD tasks demonstrating that online RL algorithms such as Group Relative Preference Optimization (GRPO) outperform offline alternatives. In the DeepCAD benchmark, our SFT model outperforms existing single-modal approaches in all three input modalities simultaneously. More importantly, after RL fine-tuning, cadrille sets new state-of-the-art on three challenging datasets, including a real-world one. Code is avaliable at https://github.com/col14m/cadrille . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_22914 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning Kolodiazhnyi, Maksim Tarasov, Denis Zhemchuzhnikov, Dmitrii Nikulin, Alexander Zisman, Ilya Vorontsova, Anna Konushin, Anton Kurenkov, Vladislav Rukhovich, Danila Computer Vision and Pattern Recognition Machine Learning Computer-Aided Design (CAD) plays a central role in engineering and manufacturing, making it possible to create precise and editable 3D models. Using a variety of sensor or user-provided data as inputs for CAD reconstruction can democratize access to design applications. However, existing methods typically focus on a single input modality, such as point clouds, images, or text, which limits their generalizability and robustness. Leveraging recent advances in vision-language models (VLM), we propose a multi-modal CAD reconstruction model that simultaneously processes all three input modalities. Inspired by large language model (LLM) training paradigms, we adopt a two-stage pipeline: supervised fine-tuning (SFT) on large-scale procedurally generated data, followed by reinforcement learning (RL) fine-tuning using online feedback, obtained programatically. Furthermore, we are the first to explore RL fine-tuning of LLMs for CAD tasks demonstrating that online RL algorithms such as Group Relative Preference Optimization (GRPO) outperform offline alternatives. In the DeepCAD benchmark, our SFT model outperforms existing single-modal approaches in all three input modalities simultaneously. More importantly, after RL fine-tuning, cadrille sets new state-of-the-art on three challenging datasets, including a real-world one. Code is avaliable at https://github.com/col14m/cadrille . |
| title | cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2505.22914 |