Cover Image

DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhu, Qiming, Cao, Jialun, Lu, Yaojie, Lin, Hongyu, Han, Xianpei, Sun, Le, Cheung, Shing-Chi
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation
by: Zhu, Qiming, et al.
Published: (2025)

EmbedAgent: Benchmarking Large Language Models in Embedded System Development
by: Xu, Ruiyang, et al.
Published: (2025)

Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
by: Zheng, Jiasheng, et al.
Published: (2024)

Combinatorial Synthesis: Scaling Code RLVR via Atomic Decomposition and Recombination
by: Zheng, Jiasheng, et al.
Published: (2026)

LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation
by: Xu, Dong, et al.
Published: (2026)

What Builds Effective In-Context Examples for Code Generation?
by: Li, Dongze, et al.
Published: (2025)

Concerned with Data Contamination? Assessing Countermeasures in Code Language Model
by: Cao, Jialun, et al.
Published: (2024)

ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models
by: Zheng, Jiasheng, et al.
Published: (2026)

When LLMs Meet API Documentation: Can Retrieval Augmentation Aid Code Generation Just as It Helps Developers?
by: Chen, Jingyi, et al.
Published: (2025)

Isolating Language-Coding from Problem-Solving: Benchmarking LLMs with PseudoEval
by: Wu, Jiarong, et al.
Published: (2025)

Can Emulating Semantic Translation Help LLMs with Code Translation? A Study Based on Pseudocode
by: Chen, Songqiang, et al.
Published: (2025)

ReuseDroid: A VLM-empowered Android UI Test Migrator Boosted by Active Feedback
by: Li, Xiaolei, et al.
Published: (2025)

Can Large Language Models Model Programs Formally?
by: Chen, Zhiyong, et al.
Published: (2026)

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
by: Cao, Jialun, et al.
Published: (2024)

MR-Adopt: Automatic Deduction of Input Transformation Function for Metamorphic Testing
by: Xu, Congying, et al.
Published: (2024)

CODECLEANER: Elevating Standards with A Robust Data Contamination Mitigation Toolkit
by: Cao, Jialun, et al.
Published: (2024)

ModelWisdom: An Integrated Toolkit for TLA+ Model Visualization, Digest and Repair
by: Chen, Zhiyong, et al.
Published: (2026)

NoCode-bench: A Benchmark for Evaluating Natural Language-Driven Feature Addition
by: Deng, Le, et al.
Published: (2025)

RulER: Automated Rule-Based Semantic Error Localization and Repair for Code Translation
by: Jin, Shuo, et al.
Published: (2025)

Automatic Build Repair for Test Cases using Incompatible Java Versions
by: Mak, Ching Hang, et al.
Published: (2024)

Enchanting Program Specification Synthesis by Large Language Models using Static Analysis and Program Verification
by: Wen, Cheng, et al.
Published: (2024)

CAM: A Causality-based Analysis Framework for Multi-Agent Code Generation Systems
by: Lyu, Zongyi, et al.
Published: (2026)

CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution
by: Xu, Ruiyang, et al.
Published: (2024)

COMET: Coverage-guided Model Generation For Deep Learning Library Testing
by: Li, Meiziniu, et al.
Published: (2022)

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
by: Zheng, Dewu, et al.
Published: (2024)

Multi-Agent Systems for Dataset Adaptation in Software Engineering: Capabilities, Limitations, and Future Directions
by: Chen, Jingyi, et al.
Published: (2025)

MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test Cases
by: Xu, Congying, et al.
Published: (2023)

Understanding and Bridging the Planner-Coder Gap: A Systematic Study on the Robustness of Multi-Agent Systems for Code Generation
by: Lyu, Zongyi, et al.
Published: (2025)

AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators
by: Chou, Jason, et al.
Published: (2025)

Rigor, Reliability, and Reproducibility Matter: A Decade-Scale Survey of 572 Code Benchmarks
by: Cao, Jialun, et al.
Published: (2025)

Word Closure-Based Metamorphic Testing for Machine Translation
by: Xie, Xiaoyuan, et al.
Published: (2023)

On the Effectiveness of Large Language Models in Domain-Specific Code Generation
by: Gu, Xiaodong, et al.
Published: (2023)

SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark
by: Yu, Boxi, et al.
Published: (2026)

iCoRe: An Iterative Correlation-Aware Retriever for Bug Reproduction Test Generation
by: Wang, Junyi, et al.
Published: (2026)

CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation
by: Pan, Ruwei, et al.
Published: (2025)

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
by: Chen, Yeheng, et al.
Published: (2026)

Understanding and Characterizing Mock Assertions in Unit Tests
by: Zhu, Hengcheng, et al.
Published: (2025)

FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
by: Dai, Dekun, et al.
Published: (2025)

Enhancing Differential Testing With LLMs For Testing Deep Learning Libraries
by: Li, Meiziniu, et al.
Published: (2024)

MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysis
by: Xu, Congying, et al.
Published: (2026)