Cover Image

Can Language Models Go Beyond Coding? Assessing the Capability of Language Models to Build Real-World Systems

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Chenyu, Zhang, Shenglin, Huang, Zeshun, Jin, Weilin, Sun, Yongqian, Pei, Dan, Zhang, Chaoyun, Lin, Qingwei, Bansal, Chetan, Rajmohan, Saravan, Ma, Minghua
Format:	Preprint
Published:	2025
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Benchmark for Language Models in Real-World System Building
by: Jin, Weilin, et al.
Published: (2026)

EvidenT: An Evidence-Preserving Framework for Iterative System-Level Package Repair
by: Zhao, Chenyu, et al.
Published: (2026)

Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detection
by: Liu, Jun, et al.
Published: (2024)

Debugging the Debuggers: Failure-Anchored Structured Recovery for Software Engineering Agents
by: Zhao, Chenyu, et al.
Published: (2026)

An Empirical Study of Production Incidents in Generative AI Cloud Services
by: Yan, Haoran, et al.
Published: (2025)

ExeCoder: Empowering Large Language Models with Executability Representation for Code Translation
by: He, Minghua, et al.
Published: (2025)

Triage in Software Engineering: A Systematic Review of Research and Practice
by: Zhao, Yongxin, et al.
Published: (2025)

Large Language Model-Brained GUI Agents: A Survey
by: Zhang, Chaoyun, et al.
Published: (2024)

Text2Grad: Reinforcement Learning from Natural Language Feedback
by: Wang, Hanyang, et al.
Published: (2025)

From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models
by: Zhang, Jue, et al.
Published: (2025)

Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4
by: Zhang, Xuchao, et al.
Published: (2024)

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
by: Gupta, Taneesh, et al.
Published: (2025)

REFA: Reference Free Alignment for multi-preference optimization
by: Gupta, Taneesh, et al.
Published: (2024)

RuAG: Learned-rule-augmented Generation for Large Language Models
by: Zhang, Yudi, et al.
Published: (2024)

Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation
by: Ding, Ruomeng, et al.
Published: (2023)

AllHands: Ask Me Anything on Large-scale Verbatim Feedback via Large Language Models
by: Zhang, Chaoyun, et al.
Published: (2024)

API Agents vs. GUI Agents: Divergence and Convergence
by: Zhang, Chaoyun, et al.
Published: (2025)

Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?
by: Zhang, Yudi, et al.
Published: (2025)

VEM: Environment-Free Exploration for Training GUI Agent with Value Environment Model
by: Zheng, Jiani, et al.
Published: (2025)

RepoLaunch: Automating Build&Test Pipeline of Code Repositories on ANY Language and ANY Platform
by: Li, Kenan, et al.
Published: (2026)

AutoAdapt: An Automated Domain Adaptation Framework for LLMs
by: Sinha, Sidharth, et al.
Published: (2026)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
by: Gupta, Taneesh, et al.
Published: (2024)

WarriorCoder: Learning from Expert Battles to Augment Code Large Language Models
by: Feng, Huawen, et al.
Published: (2024)

Enhanced Fine-Tuning of Lightweight Domain-Specific Q&A Model Based on Large Language Models
by: Zhang, Shenglin, et al.
Published: (2024)

UFO3: Weaving the Digital Agent Galaxy
by: Zhang, Chaoyun, et al.
Published: (2025)

CARMO: Dynamic Criteria Generation for Context-Aware Reward Modelling
by: Gupta, Taneesh, et al.
Published: (2024)

UFO: A UI-Focused Agent for Windows OS Interaction
by: Zhang, Chaoyun, et al.
Published: (2024)

Beyond State Consistency: Behavior Consistency in Text-Based World Models
by: Huang, Youling, et al.
Published: (2026)

Zipage: Maintain High Request Concurrency for LLM Reasoning through Compressed PagedAttention
by: Liao, Mengqi, et al.
Published: (2026)

Minerva: A Programmable Memory Test Benchmark for Language Models
by: Xia, Menglin, et al.
Published: (2025)

WarriorMath: Enhancing the Mathematical Ability of Large Language Models with a Defect-aware Framework
by: Chen, Yue, et al.
Published: (2025)

Intelligent Monitoring Framework for Cloud Services: A Data-Driven Approach
by: Srinivas, Pooja, et al.
Published: (2024)

DI-BENCH: Benchmarking Large Language Models on Dependency Inference with Testable Repositories at Scale
by: Zhang, Linghao, et al.
Published: (2025)

AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds
by: Chen, Yinfang, et al.
Published: (2025)

Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective
by: Wang, Zexin, et al.
Published: (2024)

Assessing the Code Clone Detection Capability of Large Language Models
by: Zhang, Zixian, et al.
Published: (2024)

OpsEval: A Comprehensive IT Operations Benchmark Suite for Large Language Models
by: Liu, Yuhe, et al.
Published: (2023)

AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation
by: Hu, Mengkang, et al.
Published: (2024)

Building AI Agents for Autonomous Clouds: Challenges and Design Principles
by: Shetty, Manish, et al.
Published: (2024)

Computer-Using World Model
by: Guan, Yiming, et al.
Published: (2026)