3Dify: a Framework for Procedural 3D-CG Generation Assisted by LLMs Using MCP and RAG
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911192940806144 |
|---|---|
| author | Hayashi, Shun-ichiro Mukunoki, Daichi Hoshino, Tetsuya Ohshima, Satoshi Katagiri, Takahiro |
| author_facet | Hayashi, Shun-ichiro Mukunoki, Daichi Hoshino, Tetsuya Ohshima, Satoshi Katagiri, Takahiro |
| contents | This paper proposes "3Dify," a procedural 3D computer graphics (3D-CG) generation framework utilizing Large Language Models (LLMs). The framework enables users to generate 3D-CG content solely through natural language instructions. 3Dify is built upon Dify, an open-source platform for AI application development, and incorporates several state-of-the-art LLM-related technologies such as the Model Context Protocol (MCP) and Retrieval-Augmented Generation (RAG). For 3D-CG generation support, 3Dify automates the operation of various Digital Content Creation (DCC) tools via MCP. When DCC tools do not support MCP-based interaction, the framework employs the Computer-Using Agent (CUA) method to automate Graphical User Interface (GUI) operations. Moreover, to enhance image generation quality, 3Dify allows users to provide feedback by selecting preferred images from multiple candidates. The LLM then learns variable patterns from these selections and applies them to subsequent generations. Furthermore, 3Dify supports the integration of locally deployed LLMs, enabling users to utilize custom-developed models and to reduce both time and monetary costs associated with external API calls by leveraging their own computational resources. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_04536 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | 3Dify: a Framework for Procedural 3D-CG Generation Assisted by LLMs Using MCP and RAG Hayashi, Shun-ichiro Mukunoki, Daichi Hoshino, Tetsuya Ohshima, Satoshi Katagiri, Takahiro Graphics Artificial Intelligence Computer Vision and Pattern Recognition This paper proposes "3Dify," a procedural 3D computer graphics (3D-CG) generation framework utilizing Large Language Models (LLMs). The framework enables users to generate 3D-CG content solely through natural language instructions. 3Dify is built upon Dify, an open-source platform for AI application development, and incorporates several state-of-the-art LLM-related technologies such as the Model Context Protocol (MCP) and Retrieval-Augmented Generation (RAG). For 3D-CG generation support, 3Dify automates the operation of various Digital Content Creation (DCC) tools via MCP. When DCC tools do not support MCP-based interaction, the framework employs the Computer-Using Agent (CUA) method to automate Graphical User Interface (GUI) operations. Moreover, to enhance image generation quality, 3Dify allows users to provide feedback by selecting preferred images from multiple candidates. The LLM then learns variable patterns from these selections and applies them to subsequent generations. Furthermore, 3Dify supports the integration of locally deployed LLMs, enabling users to utilize custom-developed models and to reduce both time and monetary costs associated with external API calls by leveraging their own computational resources. |
| title | 3Dify: a Framework for Procedural 3D-CG Generation Assisted by LLMs Using MCP and RAG |
| topic | Graphics Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.04536 |