Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2509.20136 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909804705873920 |
|---|---|
| author | Zhang, Wei Yang, Jack Tao, Renshuai Chai, Lingzheng Guo, Shawn Wu, Jiajun Chen, Xiaoming Cui, Ganqu Ding, Ning Xu, Xander Wei, Hu Zhou, Bowen |
| author_facet | Zhang, Wei Yang, Jack Tao, Renshuai Chai, Lingzheng Guo, Shawn Wu, Jiajun Chen, Xiaoming Cui, Ganqu Ding, Ning Xu, Xander Wei, Hu Zhou, Bowen |
| contents | Code large language models have demonstrated remarkable capabilities in programming tasks, yet current benchmarks primarily focus on single modality rather than visual game development. Most existing code-related benchmarks evaluate syntax correctness and execution accuracy, overlooking critical game-specific metrics such as playability, visual aesthetics, and user engagement that are essential for real-world deployment. To address the gap between current LLM capabilities in algorithmic problem-solving and competitive programming versus the comprehensive requirements of practical game development, we present V-GameGym, a comprehensive benchmark comprising 2,219 high-quality samples across 100 thematic clusters derived from real-world repositories, adopting a novel clustering-based curation methodology to ensure both diversity and structural completeness. Further, we introduce a multimodal evaluation framework with an automated LLM-driven pipeline for visual code synthesis using complete UI sandbox environments. Our extensive analysis reveals that V-GameGym effectively bridges the gap between code generation accuracy and practical game development workflows, providing quantifiable quality metrics for visual programming and interactive element generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20136 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | V-GameGym: Visual Game Generation for Code Large Language Models Zhang, Wei Yang, Jack Tao, Renshuai Chai, Lingzheng Guo, Shawn Wu, Jiajun Chen, Xiaoming Cui, Ganqu Ding, Ning Xu, Xander Wei, Hu Zhou, Bowen Software Engineering Code large language models have demonstrated remarkable capabilities in programming tasks, yet current benchmarks primarily focus on single modality rather than visual game development. Most existing code-related benchmarks evaluate syntax correctness and execution accuracy, overlooking critical game-specific metrics such as playability, visual aesthetics, and user engagement that are essential for real-world deployment. To address the gap between current LLM capabilities in algorithmic problem-solving and competitive programming versus the comprehensive requirements of practical game development, we present V-GameGym, a comprehensive benchmark comprising 2,219 high-quality samples across 100 thematic clusters derived from real-world repositories, adopting a novel clustering-based curation methodology to ensure both diversity and structural completeness. Further, we introduce a multimodal evaluation framework with an automated LLM-driven pipeline for visual code synthesis using complete UI sandbox environments. Our extensive analysis reveals that V-GameGym effectively bridges the gap between code generation accuracy and practical game development workflows, providing quantifiable quality metrics for visual programming and interactive element generation. |
| title | V-GameGym: Visual Game Generation for Code Large Language Models |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2509.20136 |