Cover Image

SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Miserendino, Samuel, Wang, Michele, Patwardhan, Tejal, Heidecke, Johannes
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SWE-Tester: Training Open-Source LLMs for Issue Reproduction in Real-World Repositories
by: Soni, Aditya Bharat, et al.
Published: (2026)

SWE-Arena: An Interactive Platform for Evaluating Foundation Models in Software Engineering
by: Zhao, Zhimin
Published: (2025)

Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
by: Xia, Chunqiu Steven, et al.
Published: (2025)

SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents
by: Ding, Yifeng, et al.
Published: (2026)

SWE-MiniSandbox: Container-Free Reinforcement Learning for Building Software Engineering Agents
by: Yuan, Danlong, et al.
Published: (2026)

SWE-Universe: Scale Real-World Verifiable Environments to Millions
by: Chen, Mouxiang, et al.
Published: (2026)

SWE-Exp: Experience-Driven Software Issue Resolution
by: Chen, Silin, et al.
Published: (2025)

SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents
by: Kon, Patrick Tser Jern, et al.
Published: (2026)

IDE-Bench: Evaluating Large Language Models as IDE Agents on Real-World Software Engineering Tasks
by: Mateega, Spencer, et al.
Published: (2026)

SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution
by: Li, Han, et al.
Published: (2025)

Breaking the Silence: the Threats of Using LLMs in Software Engineering
by: Sallou, June, et al.
Published: (2023)

Can LLMs Replace Manual Annotation of Software Engineering Artifacts?
by: Ahmed, Toufique, et al.
Published: (2024)

Programming with Pixels: Can Computer-Use Agents do Software Engineering?
by: Aggarwal, Pranjal, et al.
Published: (2025)

Should Code Models Learn Pedagogically? A Preliminary Evaluation of Curriculum Learning for Real-World Software Engineering Tasks
by: Khant, Kyi Shin, et al.
Published: (2025)

SWE-Next: Scalable Real-World Software Engineering Tasks for Agents
by: Liang, Jiarong, et al.
Published: (2026)

Investigating Test Overfitting on SWE-bench
by: Ahmed, Toufique, et al.
Published: (2025)

SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution
by: Raghavendra, Mohit, et al.
Published: (2026)

SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories
by: Wang, Lilin, et al.
Published: (2025)

Otter: Generating Tests from Issues to Validate SWE Patches
by: Ahmed, Toufique, et al.
Published: (2025)

GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
by: Shetty, Manish, et al.
Published: (2025)

LLMs in Coding and their Impact on the Commercial Software Engineering Landscape
by: Belozerov, Vladislav, et al.
Published: (2025)

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
by: Yang, John, et al.
Published: (2024)

Toward Training Superintelligent Software Agents through Self-Play SWE-RL
by: Wei, Yuxiang, et al.
Published: (2025)

SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?
by: Han, Tingxu, et al.
Published: (2026)

Agint: Agentic Graph Compilation for Software Engineering Agents
by: Chivukula, Abhi, et al.
Published: (2025)

SWE-Perf: Can Language Models Optimize Code Performance on Real-World Repositories?
by: He, Xinyi, et al.
Published: (2025)

SWE-World: Building Software Engineering Agents in Docker-Free Environments
by: Sun, Shuang, et al.
Published: (2026)

Software Engineering Principles for Fairer Systems: Experiments with GroupCART
by: Peng, Kewen, et al.
Published: (2025)

Large Language Models for Software Engineering: A Reproducibility Crisis
by: Siddiq, Mohammed Latif, et al.
Published: (2025)

Assessing the Use of AutoML for Data-Driven Software Engineering
by: Calefato, Fabio, et al.
Published: (2023)

SeaView: Software Engineering Agent Visual Interface for Enhanced Workflow
by: Bula, Timothy, et al.
Published: (2025)

Cataloguing Hugging Face Models to Software Engineering Activities: Automation and Findings
by: González, Alexandra, et al.
Published: (2025)

A Systematic Literature Review on the Use of Machine Learning in Software Engineering
by: Fred, Nyaga, et al.
Published: (2024)

SWE-Bench-CL: Continual Learning for Coding Agents
by: Joshi, Thomas, et al.
Published: (2025)

More Rigorous Software Engineering Would Improve Reproducibility in Machine Learning Research
by: Wolter, Moritz, et al.
Published: (2025)

Combating Toxic Language: A Review of LLM-Based Strategies for Software Engineering
by: Zhuo, Hao, et al.
Published: (2025)

Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products
by: Nahar, Nadia, et al.
Published: (2024)

Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education
by: Mojica-Hanke, Anamaria, et al.
Published: (2024)

HE-SNR: Uncovering Latent Logic via Entropy for Guiding Mid-Training on SWE-bench
by: Wang, Yueyang, et al.
Published: (2026)

CPP-UT-Bench: Can LLMs Write Complex Unit Tests in C++?
by: Bhargava, Vaishnavi, et al.
Published: (2024)