Image de couverture de livre

COMPASS: A Multi-Dimensional Benchmark for Evaluating Code Generation in Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Meaden, James, Jarosz, Michał, Jodłowski, Piotr, Melnik, Grigori
Format:	Preprint
Publié:	2025
Sujets:	Software Engineering Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
par: Padwal, Vedant
Publié: (2026)

CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
par: Yin, Wenjing, et autres
Publié: (2025)

Impact of Code Context and Prompting Strategies on Automated Unit Test Generation with Modern General-Purpose Large Language Models
par: Walczak, Jakub, et autres
Publié: (2025)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
par: Zhang, Tanghaoran, et autres
Publié: (2026)

SimdBench: Benchmarking Large Language Models for SIMD-Intrinsic Code Generation
par: He, Yibo, et autres
Publié: (2025)

Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
par: Zheng, Jiasheng, et autres
Publié: (2024)

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
par: He, Mengliang, et autres
Publié: (2025)

CrossPL: Evaluating Large Language Models on Cross Programming Language Code Generation
par: Xiong, Zhanhang, et autres
Publié: (2025)

Evaluating Large Language Models for Code Review
par: Cihan, Umut, et autres
Publié: (2025)

Evaluating Large Language Models in Code Generation: INFINITE Methodology for Defining the Inference Index
par: Christakis, Nicholas, et autres
Publié: (2025)

A Code Comprehension Benchmark for Large Language Models for Code
par: Havare, Jayant, et autres
Publié: (2025)

How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study
par: Velasco, Alejandro, et autres
Publié: (2024)

Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review
par: Paul, Debalina Ghosh, et autres
Publié: (2024)

GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks
par: Hou, Shuyang, et autres
Publié: (2024)

Task Abstention for Large Language Models in Code Generation
par: Zhou, Yanke, et autres
Publié: (2026)

Energy-Aware Code Generation with LLMs: Benchmarking Small vs. Large Language Models for Sustainable AI Programming
par: Ashraf, Humza, et autres
Publié: (2025)

ModiGen: A Large Language Model-Based Workflow for Multi-Task Modelica Code Generation
par: Xiang, Jiahui, et autres
Publié: (2025)

A Comparative Analysis of Large Language Models for Code Documentation Generation
par: Dvivedi, Shubhang Shekhar, et autres
Publié: (2023)

Benchmarking Correctness and Security in Multi-Turn Code Generation
par: Rawal, Ruchit, et autres
Publié: (2025)

TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models
par: Sun, Ruoyu, et autres
Publié: (2025)

RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation
par: Lin, Feng, et autres
Publié: (2025)

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
par: Paul, Debalina Ghosh, et autres
Publié: (2024)

Insights from Benchmarking Frontier Language Models on Web App Code Generation
par: Cui, Yi
Publié: (2024)

Output Format Biases in the Evaluation of Large Language Models for Code Translation
par: Macedo, Marcos, et autres
Publié: (2024)

Large Language Model Guided Self-Debugging Code Generation
par: Adnan, Muntasir, et autres
Publié: (2025)

Personality-Guided Code Generation Using Large Language Models
par: Guo, Yaoqi, et autres
Publié: (2024)

Bugs in Large Language Models Generated Code: An Empirical Study
par: Tambon, Florian, et autres
Publié: (2024)

An Execution-Verified Multi-Language Benchmark for Code Semantic Reasoning
par: Li, Yikun, et autres
Publié: (2026)

Software Development Life Cycle Perspective: A Survey of Benchmarks for Code Large Language Models and Agents
par: Wang, Kaixin, et autres
Publié: (2025)

Towards Advancing Code Generation with Large Language Models: A Research Roadmap
par: Jin, Haolin, et autres
Publié: (2025)

DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation
par: Zhu, Qiming, et autres
Publié: (2024)

Beyond Output Correctness: Benchmarking and Evaluating Large Language Model Reasoning in Coding Tasks
par: Li, Yuangang, et autres
Publié: (2026)

Combining Large Language Models with Static Analyzers for Code Review Generation
par: Jaoua, Imen, et autres
Publié: (2025)

CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generation
par: Gui, Ningxin, et autres
Publié: (2025)

Model-Driven Quantum Code Generation Using Large Language Models and Retrieval-Augmented Generation
par: Siavash, Nazanin, et autres
Publié: (2025)

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
par: Lian, Keke, et autres
Publié: (2025)

A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?
par: Chen, QiHong, et autres
Publié: (2024)

SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
par: Xu, Jingxuan, et autres
Publié: (2025)

Beyond Autoregression: An Empirical Study of Diffusion Large Language Models for Code Generation
par: Li, Chengze, et autres
Publié: (2025)

TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models
par: Tambon, Florian, et autres
Publié: (2024)