Cover Image

ProgramBench: Can Language Models Rebuild Programs From Scratch?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, John, Lieret, Kilian, Ma, Jeffrey, Thakkar, Parth, Pedchenko, Dmitrii, Sootla, Sten, McMilin, Emily, Yin, Pengcheng, Hou, Rui, Synnaeve, Gabriel, Yang, Diyi, Press, Ofir
Format:	Preprint
Published:	2026
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Underspecification in Language Modeling Tasks: A Causality-Informed Study of Gendered Pronoun Resolution
by: McMilin, Emily
Published: (2022)

CodeClash: Benchmarking Goal-Oriented Software Engineering
by: Yang, John, et al.
Published: (2025)

SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?
by: Yang, John, et al.
Published: (2024)

SWE-smith: Scaling Data for Software Engineering Agents
by: Yang, John, et al.
Published: (2025)

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
by: Yang, John, et al.
Published: (2024)

Toward Training Superintelligent Software Agents through Self-Play SWE-RL
by: Wei, Yuxiang, et al.
Published: (2025)

AlgoTune: Can Language Models Speed Up General-Purpose Numerical Programs?
by: Press, Ori, et al.
Published: (2025)

VideoGameBench: Can Vision-Language Models complete popular video games?
by: Zhang, Alex L., et al.
Published: (2025)

Properties of Eventually Positive Linear Input-Output Systems
by: Sootla, Aivar
Published: (2015)

AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
by: Yoran, Ori, et al.
Published: (2024)

CiteME: Can Language Models Accurately Cite Scientific Claims?
by: Press, Ori, et al.
Published: (2024)

BigO(Bench) -- Can LLMs Generate Code with Controlled Time and Space Complexity?
by: Chambon, Pierre, et al.
Published: (2025)

Automatically Generating Questions About Scratch Programs
by: Obermüller, Florian, et al.
Published: (2025)

Detecting Gender Stereotypes in Scratch Programming Tutorials
by: Graßl, Isabella, et al.
Published: (2025)

Sicheres autonomes Flugrobotersystem für den Einsatz im Produktions- und Logistikumfeld
by: Lieret, Markus
Published: (2025)

Finding Needles in Images: Can Multimodal LLMs Locate Fine Details?
by: Thakkar, Parth, et al.
Published: (2025)

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
by: Jimenez, Carlos E., et al.
Published: (2023)

WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch
by: Lu, Zimu, et al.
Published: (2025)

SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
by: Ma, Jeffrey Jian, et al.
Published: (2025)

EquiBench: Benchmarking Large Language Models' Reasoning about Program Semantics via Equivalence Checking
by: Wei, Anjiang, et al.
Published: (2025)

Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
by: Si, Chenglei, et al.
Published: (2024)

Differentiating Through a Quadratic Cone Program
by: Healey, Quill, et al.
Published: (2025)

Disproving Program Equivalence with LLMs
by: Allamanis, Miltiadis, et al.
Published: (2025)

EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities
by: Abramovich, Talor, et al.
Published: (2024)

Can LLMs Reason in the Wild with Programs?
by: Yang, Yuan, et al.
Published: (2024)

Chapter 13 Rebuilding Relationships between Data, Method, and Theories
by: Cucina, Jeffrey M., et al.
Published: (2022)

La inmigración hispana en Italia: hacia una variedad de contacto entre español e italiano
by: Milin Bonomi
Published: (2010)

Entre divergencia y acomodación: el caso de los inmigrantes hispanos en Barcelona y Milán
by: Milin Bonomi
Published: (2010)

Alonso, José Antonio y Rodolfo Gutiérrez (dir.). 2010. Emigración y lengua. El papel del español en las migraciones internacionales. Madrid: Ariel y Fundación Telefónica, 304 pp.
by: Milin Bonomi
Published: (2011)

Can prompt cusps of WIMP dark matter be detected as individual gamma-ray sources?
by: Delos, M. Sten
Published: (2023)

ScratchEval : A Multimodal Evaluation Framework for LLMs in Block-Based Programming
by: Si, Yuan, et al.
Published: (2026)

Raven: Rethinking Automated Assessment for Scratch Programs via Video-Grounded Evaluation
by: Li, Donglin, et al.
Published: (2026)

Rebuilding Syria
Published: (2019)

ProBench: Benchmarking Large Language Models in Competitive Programming
by: Yang, Lei, et al.
Published: (2025)

How Rules Represent Causal Knowledge: Causal Modeling with Abductive Logic Programs
by: Rückschloß, Kilian, et al.
Published: (2025)

Collection Development from Scratch: Supporting a New Degree Program in Construction Management
by: Reycraft, Kimberly
Published: (2020)

GeoGramBench: Benchmarking the Geometric Program Reasoning in Modern LLMs
by: Luo, Shixian, et al.
Published: (2025)

ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers?
by: Ye, Christine, et al.
Published: (2025)

Dynamic Skill Adaptation for Large Language Models
by: Chen, Jiaao, et al.
Published: (2024)

Searching for Privacy Risks in LLM Agents via Simulation
by: Zhang, Yanzhe, et al.
Published: (2025)