Cover Image

Investigating Test Overfitting on SWE-bench

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ahmed, Toufique, Ganhotra, Jatin, Shinnar, Avraham, Hirzel, Martin
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reproduction Test Generation for Java SWE Issues
by: Ahmed, Toufique, et al.
Published: (2026)

Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection
by: Ahmed, Toufique, et al.
Published: (2025)

Otter: Generating Tests from Issues to Validate SWE Patches
by: Ahmed, Toufique, et al.
Published: (2025)

TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved?
by: Ahmed, Toufique, et al.
Published: (2024)

Resolving Java Code Repository Issues with iSWE Agent
by: Ganhotra, Jatin, et al.
Published: (2026)

Can Old Tests Do New Tricks for Resolving SWE Issues?
by: Chen, Yang, et al.
Published: (2025)

Evaluating Plan Compliance in Autonomous Programming Agents
by: Liu, Shuyang, et al.
Published: (2026)

When Agents go Astray: Course-Correcting SWE Agents with PRMs
by: Gandhi, Shubham, et al.
Published: (2025)

CoDocBench: A Dataset for Code-Documentation Alignment in Software Maintenance
by: Pai, Kunal, et al.
Published: (2025)

HE-SNR: Uncovering Latent Logic via Entropy for Guiding Mid-Training on SWE-bench
by: Wang, Yueyang, et al.
Published: (2026)

Studying LLM Performance on Closed- and Open-source Data
by: Ahmed, Toufique, et al.
Published: (2024)

Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
by: Ahmed, Toufique, et al.
Published: (2023)

How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench
by: Sajadi, Amirali, et al.
Published: (2025)

SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents
by: Ding, Yifeng, et al.
Published: (2026)

Can LLMs Replace Manual Annotation of Software Engineering Artifacts?
by: Ahmed, Toufique, et al.
Published: (2024)

Trojans in Large Language Models of Code: A Critical Review through a Trigger-Based Taxonomy
by: Hussain, Aftab, et al.
Published: (2024)

SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution
by: Raghavendra, Mohit, et al.
Published: (2026)

SWE-bench Goes Live!
by: Zhang, Linghao, et al.
Published: (2025)

SWE-Arena: An Interactive Platform for Evaluating Foundation Models in Software Engineering
by: Zhao, Zhimin
Published: (2025)

Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Study
by: Wang, You, et al.
Published: (2025)

Towards Understanding What Code Language Models Learned
by: Ahmed, Toufique, et al.
Published: (2023)

SWE-Tester: Training Open-Source LLMs for Issue Reproduction in Real-World Repositories
by: Soni, Aditya Bharat, et al.
Published: (2026)

Calibration and Correctness of Language Models for Code
by: Spiess, Claudio, et al.
Published: (2024)

SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?
by: Miserendino, Samuel, et al.
Published: (2025)

Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generation
by: Haider, Md. Asif, et al.
Published: (2024)

SWE-Bench-CL: Continual Learning for Coding Agents
by: Joshi, Thomas, et al.
Published: (2025)

SWE-Exp: Experience-Driven Software Issue Resolution
by: Chen, Silin, et al.
Published: (2025)

Timing Analysis Agent: Autonomous Multi-Corner Multi-Mode (MCMM) Timing Debugging with Timing Debug Relation Graph
by: Nainani, Jatin, et al.
Published: (2025)

Exploring the Potential of Conversational Test Suite Based Program Repair on SWE-bench
by: Cheshkov, Anton, et al.
Published: (2024)

SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution
by: Li, Han, et al.
Published: (2025)

SWE-MiniSandbox: Container-Free Reinforcement Learning for Building Software Engineering Agents
by: Yuan, Danlong, et al.
Published: (2026)

Beyond Verifiable Rewards: Rubric-Based GRM for Reinforced Fine-Tuning SWE Agents
by: Huang, Jiawei, et al.
Published: (2026)

Representing Prompting Patterns with PDL: Compliance Agent Case Study
by: Vaziri, Mandana, et al.
Published: (2025)

SWE-Adept: An LLM-Based Agentic Framework for Deep Codebase Analysis and Structured Issue Resolution
by: He, Kang, et al.
Published: (2026)

Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving
by: Zan, Daoguang, et al.
Published: (2025)

R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
by: Jain, Naman, et al.
Published: (2025)

GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
by: Shetty, Manish, et al.
Published: (2025)

SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning
by: Peng, Jinjun, et al.
Published: (2026)

SWE-bench-java: A GitHub Issue Resolving Benchmark for Java
by: Zan, Daoguang, et al.
Published: (2024)

SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?
by: Yang, John, et al.
Published: (2024)