Cover Image

SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Jing, Guo, Lianghong, Wang, Yanlin, Liu, Mingwei, Chen, Jiachi, Ma, Yuchi, Shi, Ensheng, Zhuo, Terry Yue, Zhang, Hongyu, Zheng, Zibin
Format:	Preprint
Published:	2025
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

When to Stop? Towards Efficient Code Generation in LLMs with Excess Token Prevention
by: Guo, Lianghong, et al.
Published: (2024)

HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generation
by: Zheng, Dewu, et al.
Published: (2024)

OmniGIRL: A Multilingual and Multimodal Benchmark for GitHub Issue Resolution
by: Guo, Lianghong, et al.
Published: (2025)

ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation
by: Liu, Sicong, et al.
Published: (2026)

EffiReasonTrans: RL-Optimized Reasoning for Code Translation
by: Wang, Yanlin, et al.
Published: (2025)

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
by: Zheng, Dewu, et al.
Published: (2024)

You Augment Me: Exploring ChatGPT-based Data Augmentation for Semantic Code Search
by: Wang, Yanlin, et al.
Published: (2024)

AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion
by: Jiang, Tianyue, et al.
Published: (2026)

DRAINCODE: Stealthy Energy Consumption Attacks on Retrieval-Augmented Code Generation via Context Poisoning
by: Wang, Yanlin, et al.
Published: (2026)

Towards an Understanding of Large Language Models in Software Engineering Tasks
by: Zheng, Zibin, et al.
Published: (2023)

Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models
by: Wang, Yanlin, et al.
Published: (2024)

RLCoder: Reinforcement Learning for Repository-Level Code Completion
by: Wang, Yanlin, et al.
Published: (2024)

Towards an Understanding of Context Utilization in Code Intelligence
by: Wang, Yanlin, et al.
Published: (2025)

A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends
by: Zheng, Zibin, et al.
Published: (2023)

SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks
by: Guo, Lianghong, et al.
Published: (2025)

Agents in Software Engineering: Survey, Landscape, and Vision
by: Wang, Yanlin, et al.
Published: (2024)

RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
by: Wang, Yanlin, et al.
Published: (2026)

CoSQA+: Pioneering the Multi-Choice Code Search Benchmark with Test-Driven Agents
by: Gong, Jing, et al.
Published: (2024)

Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Survey
by: Li, Caihua, et al.
Published: (2026)

RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
by: Wang, Yanli, et al.
Published: (2024)

Efficiently Detecting Reentrancy Vulnerabilities in Complex Smart Contracts
by: Wang, Zexu, et al.
Published: (2024)

An Empirical Study of Agent Developer Practices in AI Agent Frameworks
by: Wang, Yanlin, et al.
Published: (2025)

SparseCoder: Identifier-Aware Sparse Transformer for File-Level Code Summarization
by: Wang, Yanlin, et al.
Published: (2024)

LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation
by: Zhang, Ziyao, et al.
Published: (2024)

RustRepoTrans: Repository-level Code Translation Benchmark Targeting Rust
by: Ou, Guangsheng, et al.
Published: (2024)

Knowledge-Graph-Driven Data Synthesis for Low-Resource Software Development: A HarmonyOS Case Study
by: Liu, Mingwei, et al.
Published: (2025)

A Hierarchical and Evolvable Benchmark for Fine-Grained Code Instruction Following with Multi-Turn Feedback
by: Duan, Guoliang, et al.
Published: (2025)

Build-Aware Incremental C-to-Rust Migration via Skeleton-First Translation and Historical Knowledge Reuse
by: Wang, Shengbo, et al.
Published: (2026)

KTester: Leveraging Domain and Testing Knowledge for More Effective LLM-based Test Generation
by: Li, Anji, et al.
Published: (2025)

What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond
by: Gu, Wenchao, et al.
Published: (2025)

A Preliminary Study on the Robustness of Code Generation by Large Language Models
by: Li, Zike, et al.
Published: (2025)

SECRET: Towards Scalable and Efficient Code Retrieval via Segmented Deep Hashing
by: Gu, Wenchao, et al.
Published: (2024)

Architecture-Aware Multi-Design Generation for Repository-Level Feature Addition
by: Liu, Mingwei, et al.
Published: (2026)

Dynamic analysis enhances issue resolution
by: Liu, Mingwei, et al.
Published: (2026)

RustEvo^2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation
by: Liang, Linxi, et al.
Published: (2025)

Identifying Smart Contract Security Issues in Code Snippets from Stack Overflow
by: Chen, Jiachi, et al.
Published: (2024)

Definition and Detection of Centralization Defects in Smart Contracts
by: Lin, Zewei, et al.
Published: (2024)

Are Decoder-Only Large Language Models the Silver Bullet for Code Search?
by: Chen, Yuxuan, et al.
Published: (2024)

An Empirical Study on Low-Code Programming using Traditional vs Large Language Model Support
by: Liu, Yongkun, et al.
Published: (2024)

RepoDoc: A Knowledge Graph-Based Framework to Automatic Documentation Generation and Incremental Updates
by: Xu, Dong, et al.
Published: (2026)