Cover Image

Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zheng, Jiasheng, Cao, Boxi, Ma, Zhengzhao, Pan, Ruotong, Lin, Hongyu, Lu, Yaojie, Han, Xianpei, Sun, Le
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models
by: Zheng, Jiasheng, et al.
Published: (2026)

Combinatorial Synthesis: Scaling Code RLVR via Atomic Decomposition and Recombination
by: Zheng, Jiasheng, et al.
Published: (2026)

DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation
by: Zhu, Qiming, et al.
Published: (2024)

Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation
by: Zhu, Qiming, et al.
Published: (2025)

EmbedAgent: Benchmarking Large Language Models in Embedded System Development
by: Xu, Ruiyang, et al.
Published: (2025)

Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models
by: Wang, Yanlin, et al.
Published: (2024)

LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation
by: Xu, Dong, et al.
Published: (2026)

CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation
by: Pan, Ruwei, et al.
Published: (2025)

HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generation
by: Zheng, Dewu, et al.
Published: (2024)

Beyond Output Correctness: Benchmarking and Evaluating Large Language Model Reasoning in Coding Tasks
by: Li, Yuangang, et al.
Published: (2026)

A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends
by: Zheng, Zibin, et al.
Published: (2023)

Multi-Facet Counterfactual Learning for Content Quality Evaluation
by: Zheng, Jiasheng, et al.
Published: (2024)

Modularization is Better: Effective Code Generation with Modular Prompting
by: Pan, Ruwei, et al.
Published: (2025)

Do Not Treat Code as Natural Language: Implications for Repository-Level Code Generation and Beyond
by: Le-Anh, Minh, et al.
Published: (2026)

NoCode-bench: A Benchmark for Evaluating Natural Language-Driven Feature Addition
by: Deng, Le, et al.
Published: (2025)

FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
by: Dai, Dekun, et al.
Published: (2025)

SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA
by: Zhang, Jing, et al.
Published: (2025)

SRLCG: Self-Rectified Large-Scale Code Generation with Multidimensional Chain-of-Thought and Dynamic Backtracking
by: Ma, Hongru, et al.
Published: (2025)

Large Language Model Unlearning for Source Code
by: Jiang, Xue, et al.
Published: (2025)

Beyond Language Barriers: Multi-Agent Coordination for Multi-Language Code Generation
by: Moumoula, Micheline Bénédicte, et al.
Published: (2025)

CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation
by: Yan, Kaiwen, et al.
Published: (2025)

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
by: Zheng, Dewu, et al.
Published: (2024)

A Multi-Language Object-Oriented Programming Benchmark for Large Language Models
by: Wang, Shuai, et al.
Published: (2025)

Ensembling Large Language Models for Code Vulnerability Detection: An Empirical Evaluation
by: Sun, Zhihong, et al.
Published: (2025)

CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
by: Yin, Wenjing, et al.
Published: (2025)

Benchmarking Correctness and Security in Multi-Turn Code Generation
by: Rawal, Ruchit, et al.
Published: (2025)

DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models
by: Chen, Yunnong, et al.
Published: (2025)

Can Large Language Models Serve as Evaluators for Code Summarization?
by: Wu, Yang, et al.
Published: (2024)

Beyond Code Generation: Assessing Code LLM Maturity with Postconditions
by: He, Fusen, et al.
Published: (2024)

On the Effectiveness of Large Language Models in Domain-Specific Code Generation
by: Gu, Xiaodong, et al.
Published: (2023)

CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs
by: Manh, Dung Nguyen, et al.
Published: (2024)

QuanBench: Benchmarking Quantum Code Generation with Large Language Models
by: Guo, Xiaoyu, et al.
Published: (2025)

Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models
by: Yan, Xinru, et al.
Published: (2026)

Hybrid Automated Program Repair by Combining Large Language Models and Program Analysis
by: Li, Fengjie, et al.
Published: (2024)

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
by: Padwal, Vedant
Published: (2026)

AdaCoder: An Adaptive Planning and Multi-Agent Framework for Function-Level Code Generation
by: Zhu, Yueheng, et al.
Published: (2025)

Benchmarking Large Language Models for Multi-Language Software Vulnerability Detection
by: Zhang, Ting, et al.
Published: (2025)

Beyond Strict Rules: Assessing the Effectiveness of Large Language Models for Code Smell Detection
by: Souza, Saymon, et al.
Published: (2026)

Can Language Models Go Beyond Coding? Assessing the Capability of Language Models to Build Real-World Systems
by: Zhao, Chenyu, et al.
Published: (2025)

Empirical Evaluation of Large Language Models in Automated Program Repair
by: Sun, Jiajun, et al.
Published: (2025)