Copertina

SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xiang, Yanzheng, Yan, Hanqi, Ouyang, Shuyin, Gui, Lin, He, Yulan
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence Multiagent Systems Software Engineering
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces
di: Feng, Yukang, et al.
Pubblicazione: (2026)

SWE-WebDevBench: Evaluating Coding Agent Application Platforms as Virtual Software Agencies
di: Saxena, Siddhant, et al.
Pubblicazione: (2026)

Sibyl-AutoResearch: Autonomous Research Needs Self-Evolving Trial-and-Error Harnesses, Not Paper Generators
di: Wang, Chengcheng, et al.
Pubblicazione: (2026)

Gecko: A Simulation Environment with Stateful Feedback for Refining Agent Tool Calls
di: Zhang, Zeyu, et al.
Pubblicazione: (2026)

An Executable Benchmarking Suite for Tool-Using Agents
di: Zhong, Zhiqing, et al.
Pubblicazione: (2026)

Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
di: Zhou, Chenyu, et al.
Pubblicazione: (2026)

ABMax: A JAX-based Agent-based Modeling Framework
di: Chaturvedi, Siddharth, et al.
Pubblicazione: (2025)

Real-Time BDI Agents: a model and its implementation
di: Traldi, Andrea, et al.
Pubblicazione: (2022)

CodeCureAgent: Automatic Classification and Repair of Static Analysis Warnings
di: Joos, Pascal, et al.
Pubblicazione: (2025)

Bridging the Prototype-Production Gap: A Multi-Agent System for Notebooks Transformation
di: Elhashemy, Hanya, et al.
Pubblicazione: (2025)

Cognitive Agents Powered by Large Language Models for Agile Software Project Management
di: Cinkusz, Konrad, et al.
Pubblicazione: (2025)

REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement
di: Pabba, Anvith, et al.
Pubblicazione: (2025)

Fairness in Multi-Agent Systems for Software Engineering: An SDLC-Oriented Rapid Review
di: Yang-Smith, Corey, et al.
Pubblicazione: (2026)

Analyzing Code Injection Attacks on LLM-based Multi-Agent Systems in Software Development
di: Bowers, Brian, et al.
Pubblicazione: (2025)

IACT: A Self-Organizing Recursive Model for General AI Agents: A Technical White Paper on the Architecture Behind kragent.ai
di: Lu, Pengju
Pubblicazione: (2025)

ToolRosella: Translating Code Repositories into Standardized Tools for Scientific Agents
di: Di, Shimin, et al.
Pubblicazione: (2026)

A Research Agenda on Agents and Software Engineering: Outcomes from the Rio A2SE Seminar
di: Taibi, Davide, et al.
Pubblicazione: (2026)

Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Research
di: Krawiecka, Klaudia, et al.
Pubblicazione: (2025)

WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing
di: Kong, Fanheng, et al.
Pubblicazione: (2026)

BootstrapAgent: Distilling Repository Setup into Reusable Agent Knowledge
di: Fu, Sihan, et al.
Pubblicazione: (2026)

SemAgent: A Semantics Aware Program Repair Agent
di: Pabba, Anvith, et al.
Pubblicazione: (2025)

DSCodeBench: A Realistic Benchmark for Data Science Code Generation
di: Ouyang, Shuyin, et al.
Pubblicazione: (2025)

ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologies
di: Gandhi, Shubham, et al.
Pubblicazione: (2025)

Agent Behavioral Contracts: Formal Specification and Runtime Enforcement for Reliable Autonomous AI Agents
di: Bhardwaj, Varun Pratap
Pubblicazione: (2026)

Trajectory Supervision for Continual Tool-Use Learning in LLMs
di: Reddy, Vishnu Vardhan, et al.
Pubblicazione: (2026)

Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineering
di: Ronanki, Krishna
Pubblicazione: (2025)

Modality-Native Routing in Agent-to-Agent Networks: A Multimodal A2A Protocol Extension
di: Srinivasan, Vasundra
Pubblicazione: (2026)

AgentGit: A Version Control Framework for Reliable and Scalable LLM-Powered Multi-Agent Systems
di: Li, Yang, et al.
Pubblicazione: (2025)

Affordance Representation and Recognition for Autonomous Agents
di: Gidey, Habtom Kahsay, et al.
Pubblicazione: (2025)

Lessons Learned: A Multi-Agent Framework for Code LLMs to Learn and Improve
di: Liu, Yuanzhe, et al.
Pubblicazione: (2025)

ChatDev: Communicative Agents for Software Development
di: Qian, Chen, et al.
Pubblicazione: (2023)

Spec Kit Agents: Context-Grounded Agentic Workflows
di: Taghavi, Pardis, et al.
Pubblicazione: (2026)

The Lifecycle Workbench -- A Configurable Framework for Digitized Product Maintenance Services
di: Briechle, Dominique, et al.
Pubblicazione: (2025)

Deterministic vs. LLM-Controlled Orchestration for COBOL-to-Python Modernization
di: Lwin, Naing Oo, et al.
Pubblicazione: (2026)

Sherlock: Reliable and Efficient Agentic Workflow Execution
di: Ro, Yeonju, et al.
Pubblicazione: (2025)

A Generic Modelling Framework for Last-Mile Delivery Systems
di: Gürcan, Önder, et al.
Pubblicazione: (2025)

AutoFSM: A Multi-agent Framework for FSM Code Generation with IR and SystemC-Based Testing
di: Luo, Qiuming, et al.
Pubblicazione: (2025)

The Hidden Bloat in Machine Learning Systems
di: Zhang, Huaifeng, et al.
Pubblicazione: (2025)

Runtime Composition in Dynamic System of Systems: A Systematic Review of Challenges, Solutions, Tools, and Evaluation Methods
di: Ashfaq, Muhammad, et al.
Pubblicazione: (2025)

A Step Towards a Universal Method for Modeling and Implementing Cross-Organizational Business Processes
di: Zeisler, Gerhard, et al.
Pubblicazione: (2024)