Cover Image

ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Code

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hua, Tianyu, Hua, Harper, Xiang, Violet, Klieger, Benjamin, Truong, Sang T., Liang, Weixin, Sun, Fan-Yun, Haber, Nick
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing
by: Fein, Daniel, et al.
Published: (2025)

Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning
by: Xiang, Violet, et al.
Published: (2025)

From Replication to Redesign: Exploring Pairwise Comparisons for LLM-Based Peer Review
by: Zhang, Yaohui, et al.
Published: (2025)

OSS-Bench: Benchmark Generator for Coding LLMs
by: Jiang, Yuancheng, et al.
Published: (2025)

ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams
by: Klieger, Benjamin, et al.
Published: (2024)

SWE-Bench+: Enhanced Coding Benchmark for LLMs
by: Aleithan, Reem, et al.
Published: (2024)

ResearchEnvBench: Benchmarking Agents on Environment Synthesis for Research Code Execution
by: Wang, Yubang, et al.
Published: (2026)

The Entropy Characterization of Quantum MDS Codes
by: Sun, Hua
Published: (2025)

One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Models
by: Fein, Daniel, et al.
Published: (2026)

Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
by: Cross, Logan, et al.
Published: (2024)

PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization
by: Jing, Huihao, et al.
Published: (2026)

SensorBench: Benchmarking LLMs in Coding-Based Sensor Processing
by: Quan, Pengrui, et al.
Published: (2024)

Code2Bench: Scaling Source and Rigor for Dynamic Benchmark Construction
by: Zhang, Zhe, et al.
Published: (2025)

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation
by: Chen, Zaoyu, et al.
Published: (2026)

Prediction of Item Difficulty for Reading Comprehension Items by Creation of Annotated Item Repository
by: Kapoor, Radhika, et al.
Published: (2025)

JMigBench: A Benchmark for Evaluating LLMs on Source Code Migration (Java 8 to Java 11)
by: Amin, Nishil, et al.
Published: (2026)

SKA-Bench: A Fine-Grained Benchmark for Evaluating Structured Knowledge Understanding of LLMs
by: Liu, Zhiqiang, et al.
Published: (2025)

SciCode: A Research Coding Benchmark Curated by Scientists
by: Tian, Minyang, et al.
Published: (2024)

CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Models
by: Zhang, Alexander, et al.
Published: (2025)

FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
by: Li, Wei, et al.
Published: (2025)

Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images
by: Wu, Shengguang, et al.
Published: (2025)

HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation
by: Chen, Qirui, et al.
Published: (2026)

EVM-QuestBench: An Execution-Grounded Benchmark for Natural-Language Transaction Code Generation
by: Yang, Pei, et al.
Published: (2026)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
by: Jiang, Hongchao, et al.
Published: (2025)

RepoMod-Bench: A Benchmark for Code Repository Modernization via Implementation-Agnostic Testing
by: Li, Xuefeng, et al.
Published: (2026)

AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
by: Duston, Titouan, et al.
Published: (2025)

SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks
by: Orlanski, Gabriel, et al.
Published: (2026)

EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems
by: Zhang, Wentao, et al.
Published: (2026)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

FEM-Bench: A Structured Scientific Reasoning Benchmark for Evaluating Code-Generating LLMs
by: Mohammadzadeh, Saeed, et al.
Published: (2025)

FeatureBench: Benchmarking Agentic Coding for Complex Feature Development
by: Zhou, Qixing, et al.
Published: (2026)

SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers
by: Xiang, Yanzheng, et al.
Published: (2025)

A Novel Ensemble Machine Learning Approach for Interpretable Modeling, Feature Extraction and Selection With Applications to Medical and Biomedical Signals and Data
by: Bo Sun, et al.
Published: (2026)

CodeRefine: A Pipeline for Enhancing LLM-Generated Code Implementations of Research Papers
by: Trofimova, Ekaterina, et al.
Published: (2024)

LongCodeBench: Evaluating Coding LLMs at 1M Context Windows
by: Rando, Stefano, et al.
Published: (2025)

AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators
by: Chou, Jason, et al.
Published: (2025)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

CodeBenchGen: Creating Scalable Execution-based Code Generation Benchmarks
by: Xie, Yiqing, et al.
Published: (2024)

How Quantization Impacts Privacy Risk on LLMs for Code?
by: Haque, Md Nazmul, et al.
Published: (2025)

CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
by: Guo, Jiawei, et al.
Published: (2024)