Training Language Models to Generate Quality Code with Program Analysis Feedback
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912400616194048 |
|---|---|
| author | Yao, Feng Wang, Zilong Liu, Liyuan Cui, Junxia Zhong, Li Fu, Xiaohan Mai, Haohui Krishnan, Vish Gao, Jianfeng Shang, Jingbo |
| author_facet | Yao, Feng Wang, Zilong Liu, Liyuan Cui, Junxia Zhong, Li Fu, Xiaohan Mai, Haohui Krishnan, Vish Gao, Jianfeng Shang, Jingbo |
| contents | Code generation with large language models (LLMs), often termed vibe coding, is increasingly adopted in production but fails to ensure code quality, particularly in security (e.g., SQL injection vulnerabilities) and maintainability (e.g., missing type annotations). Existing methods, such as supervised fine-tuning and rule-based post-processing, rely on labor-intensive annotations or brittle heuristics, limiting their scalability and effectiveness. We propose REAL, a reinforcement learning framework that incentivizes LLMs to generate production-quality code using program analysis-guided feedback. Specifically, REAL integrates two automated signals: (1) program analysis detecting security or maintainability defects and (2) unit tests ensuring functional correctness. Unlike prior work, our framework is prompt-agnostic and reference-free, enabling scalable supervision without manual intervention. Experiments across multiple datasets and model scales demonstrate that REAL outperforms state-of-the-art methods in simultaneous assessments of functionality and code quality. Our work bridges the gap between rapid prototyping and production-ready code, enabling LLMs to deliver both speed and quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_22704 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Training Language Models to Generate Quality Code with Program Analysis Feedback Yao, Feng Wang, Zilong Liu, Liyuan Cui, Junxia Zhong, Li Fu, Xiaohan Mai, Haohui Krishnan, Vish Gao, Jianfeng Shang, Jingbo Computation and Language Artificial Intelligence Code generation with large language models (LLMs), often termed vibe coding, is increasingly adopted in production but fails to ensure code quality, particularly in security (e.g., SQL injection vulnerabilities) and maintainability (e.g., missing type annotations). Existing methods, such as supervised fine-tuning and rule-based post-processing, rely on labor-intensive annotations or brittle heuristics, limiting their scalability and effectiveness. We propose REAL, a reinforcement learning framework that incentivizes LLMs to generate production-quality code using program analysis-guided feedback. Specifically, REAL integrates two automated signals: (1) program analysis detecting security or maintainability defects and (2) unit tests ensuring functional correctness. Unlike prior work, our framework is prompt-agnostic and reference-free, enabling scalable supervision without manual intervention. Experiments across multiple datasets and model scales demonstrate that REAL outperforms state-of-the-art methods in simultaneous assessments of functionality and code quality. Our work bridges the gap between rapid prototyping and production-ready code, enabling LLMs to deliver both speed and quality. |
| title | Training Language Models to Generate Quality Code with Program Analysis Feedback |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.22704 |