QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909550889664512 |
|---|---|
| author | Hu, Yaojie Zhou, Qiang Chen, Qihong Li, Xiaopeng Liu, Linbo Zhang, Dejiao Kachroo, Amit Oz, Talha Tripp, Omer |
| author_facet | Hu, Yaojie Zhou, Qiang Chen, Qihong Li, Xiaopeng Liu, Linbo Zhang, Dejiao Kachroo, Amit Oz, Talha Tripp, Omer |
| contents | We introduce QualityFlow, a dynamic agentic workflow for program synthesis. Given the English description of a programming problem and a set of unit tests, the model's goal is to synthesize the correct program that solves the problem and passes the tests. QualityFlow includes large language model (LLM) agents resembling a software development team, including code generation, testing, and self-debugging. We propose the LLM Quality Checker, which explicitly "imagines" whether the synthesized programs' execution would conform to the unit tests. The Quality Checks dynamically control the workflow, including actions to submit the final answer, clarify the problem statement, and revert previous workflow steps. Our experiments show that the Quality Checker can precisely accept any correct program, mitigate faulty synthesized tests, and prevent potential workflow deviation. QualityFlow establishes the state-of-the-art results on four program synthesis benchmarks: MBPP, HumanEval, and stricter evaluations from MBPP-EvalPlus and HumanEval-EvalPlus. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_17167 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks Hu, Yaojie Zhou, Qiang Chen, Qihong Li, Xiaopeng Liu, Linbo Zhang, Dejiao Kachroo, Amit Oz, Talha Tripp, Omer Software Engineering Artificial Intelligence We introduce QualityFlow, a dynamic agentic workflow for program synthesis. Given the English description of a programming problem and a set of unit tests, the model's goal is to synthesize the correct program that solves the problem and passes the tests. QualityFlow includes large language model (LLM) agents resembling a software development team, including code generation, testing, and self-debugging. We propose the LLM Quality Checker, which explicitly "imagines" whether the synthesized programs' execution would conform to the unit tests. The Quality Checks dynamically control the workflow, including actions to submit the final answer, clarify the problem statement, and revert previous workflow steps. Our experiments show that the Quality Checker can precisely accept any correct program, mitigate faulty synthesized tests, and prevent potential workflow deviation. QualityFlow establishes the state-of-the-art results on four program synthesis benchmarks: MBPP, HumanEval, and stricter evaluations from MBPP-EvalPlus and HumanEval-EvalPlus. |
| title | QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2501.17167 |