Cover Image

GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shetty, Manish, Jain, Naman, Liu, Jinjian, Kethanaboyina, Vijay, Sen, Koushik, Stoica, Ion
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
by: Jain, Naman, et al.
Published: (2025)

Challenges and Paths Towards AI for Software Engineering
by: Gu, Alex, et al.
Published: (2025)

Syzygy: Dual Code-Test C to (safe) Rust Translation using LLMs and Dynamic Analysis
by: Shetty, Manish, et al.
Published: (2024)

SWE-smith: Scaling Data for Software Engineering Agents
by: Yang, John, et al.
Published: (2025)

SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks
by: Adamenko, Pavel, et al.
Published: (2025)

SWE-Next: Scalable Real-World Software Engineering Tasks for Agents
by: Liang, Jiarong, et al.
Published: (2026)

SWE-AGI: Benchmarking Specification-Driven Software Construction with MoonBit in the Era of Autonomous Agents
by: Zhang, Zhirui, et al.
Published: (2026)

SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration
by: Chen, Jialong, et al.
Published: (2026)

Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents
by: Yang, Zonghan, et al.
Published: (2025)

SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?
by: Yang, John, et al.
Published: (2024)

Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering
by: Zeng, Guangtao, et al.
Published: (2025)

Toward Training Superintelligent Software Agents through Self-Play SWE-RL
by: Wei, Yuxiang, et al.
Published: (2025)

Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
by: Xia, Chunqiu Steven, et al.
Published: (2025)

APEX-SWE
by: Kottamasu, Abhi, et al.
Published: (2026)

Specifications: The missing link to making the development of LLM systems an engineering discipline
by: Stoica, Ion, et al.
Published: (2024)

SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution
by: Wei, Yuxiang, et al.
Published: (2025)

SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades
by: Lam, Man Ho, et al.
Published: (2026)

TOM-SWE: User Mental Modeling For Software Engineering Agents
by: Zhou, Xuhui, et al.
Published: (2025)

SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints
by: Fan, Zhiyu, et al.
Published: (2025)

SWE-bench Goes Live!
by: Zhang, Linghao, et al.
Published: (2025)

SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents
by: Kon, Patrick Tser Jern, et al.
Published: (2026)

Dissecting the SWE-Bench Leaderboards: Profiling Submitters and Architectures of LLM- and Agent-Based Repair Systems
by: Martinez, Matias, et al.
Published: (2025)

SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks
by: Zeng, Yucheng, et al.
Published: (2026)

Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving
by: Zan, Daoguang, et al.
Published: (2025)

SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents
by: Badertdinov, Ibragim, et al.
Published: (2025)

daVinci-Env: Open SWE Environment Synthesis at Scale
by: Fu, Dayuan, et al.
Published: (2026)

AgentLens: Revealing The Lucky Pass Problem in SWE-Agent Evaluation
by: Sahoo, Priyam, et al.
Published: (2026)

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
by: Jain, Naman, et al.
Published: (2024)

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
by: Yang, John, et al.
Published: (2024)

SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
by: Deng, Xiang, et al.
Published: (2025)

OmniCode: A Benchmark for Evaluating Software Engineering Agents
by: Sonwane, Atharv, et al.
Published: (2026)

SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?
by: Han, Tingxu, et al.
Published: (2026)

SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories
by: Wang, Lilin, et al.
Published: (2025)

Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
by: Kuang, Jiayi, et al.
Published: (2025)

SWE-bench-java: A GitHub Issue Resolving Benchmark for Java
by: Zan, Daoguang, et al.
Published: (2024)

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future
by: Jin, Haolin, et al.
Published: (2024)

BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills
by: Sonwane, Atharv, et al.
Published: (2025)

Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation
by: Garg, Spandan, et al.
Published: (2025)

SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents
by: Ding, Yifeng, et al.
Published: (2026)

SWE-Fuse: Empowering Software Agents via Issue-free Trajectory Learning and Entropy-aware RLVR Training
by: Wen, Xin-Cheng, et al.
Published: (2026)