Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.14445 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918501154816000 |
|---|---|
| author | He, Runyuan Mang, Qiuyang Zhou, Shang Liu, Kaiyuan Li, Hanchen Mao, Huanzhi Zhang, Qizheng Li, Zerui Peng, Bo Cheng, Lufeng Fu, Tianfu Wang, Yichuan Chai, Wenhao Shang, Jingbo Dimakis, Alex Gonzalez, Joseph E. Cheung, Alvin |
| author_facet | He, Runyuan Mang, Qiuyang Zhou, Shang Liu, Kaiyuan Li, Hanchen Mao, Huanzhi Zhang, Qizheng Li, Zerui Peng, Bo Cheng, Lufeng Fu, Tianfu Wang, Yichuan Chai, Wenhao Shang, Jingbo Dimakis, Alex Gonzalez, Joseph E. Cheung, Alvin |
| contents | Many real-world coding challenges are open-ended and admit no known optimal solution. Yet, recent progress in LLM coding has focused on well-defined tasks such as feature implementation, bug fixing, and competitive programming. Open-ended coding remains a weak spot for LLMs, largely because open-ended training problems are scarce and expensive to construct. Our goal is to synthesize open-ended coding problems at scale to train stronger LLM coders. We introduce FrontierSmith, an automated system for iteratively evolving open-ended problems from existing closed-ended coding tasks. Starting from competitive programming problems, FrontierSmith generates candidate open-ended variants by changing the problems'goals, restricting outputs, and generalizing inputs. It then uses a quantitative idea divergence metric to select problems that elicit genuinely diverse approaches from different solvers. Agents then generate test cases and verifiers for the surviving candidates. On two open-ended coding benchmarks, training on our synthesized data yields substantial gains over the base models: Qwen3.5-9B improves by +8.82 score on FrontierCS and +306.36 (Elo-rating-based performance) on ALE-bench; Qwen3.5-27B improves by +12.12 and +309.12, respectively. The synthesized problems also make agents take more turns and use more tokens, similar to human-curated ones, suggesting that closed-ended seeds can be a practical starting point for long-horizon coding data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_14445 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale He, Runyuan Mang, Qiuyang Zhou, Shang Liu, Kaiyuan Li, Hanchen Mao, Huanzhi Zhang, Qizheng Li, Zerui Peng, Bo Cheng, Lufeng Fu, Tianfu Wang, Yichuan Chai, Wenhao Shang, Jingbo Dimakis, Alex Gonzalez, Joseph E. Cheung, Alvin Machine Learning Many real-world coding challenges are open-ended and admit no known optimal solution. Yet, recent progress in LLM coding has focused on well-defined tasks such as feature implementation, bug fixing, and competitive programming. Open-ended coding remains a weak spot for LLMs, largely because open-ended training problems are scarce and expensive to construct. Our goal is to synthesize open-ended coding problems at scale to train stronger LLM coders. We introduce FrontierSmith, an automated system for iteratively evolving open-ended problems from existing closed-ended coding tasks. Starting from competitive programming problems, FrontierSmith generates candidate open-ended variants by changing the problems'goals, restricting outputs, and generalizing inputs. It then uses a quantitative idea divergence metric to select problems that elicit genuinely diverse approaches from different solvers. Agents then generate test cases and verifiers for the surviving candidates. On two open-ended coding benchmarks, training on our synthesized data yields substantial gains over the base models: Qwen3.5-9B improves by +8.82 score on FrontierCS and +306.36 (Elo-rating-based performance) on ALE-bench; Qwen3.5-27B improves by +12.12 and +309.12, respectively. The synthesized problems also make agents take more turns and use more tokens, similar to human-curated ones, suggesting that closed-ended seeds can be a practical starting point for long-horizon coding data. |
| title | FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.14445 |