Cover Image

CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Xinchen, Gao, Pengfei, Peng, Chao, Hu, Ruida, Gao, Cuiyun
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Repo2Run: Automated Building Executable Environment for Code Repository at Scale
by: Hu, Ruida, et al.
Published: (2025)

AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions
by: Wang, Xinchen, et al.
Published: (2024)

Evaluating Repository-level Software Documentation via Question Answering and Feature-Driven Development
by: Wang, Xinchen, et al.
Published: (2026)

Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
by: Hu, Ruida, et al.
Published: (2026)

CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering
by: Hu, Ruida, et al.
Published: (2024)

Benchmarking LLMs for Fine-Grained Code Review with Enriched Context in Practice
by: Hu, Ruida, et al.
Published: (2025)

More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents
by: Gao, Pengfei, et al.
Published: (2025)

RepoMasterEval: Evaluating Code Completion via Real-World Repositories
by: Wu, Qinyun, et al.
Published: (2024)

Search-Based LLMs for Code Optimization
by: Gao, Shuzheng, et al.
Published: (2024)

MarsCode Agent: AI-native Automated Bug Fixing
by: Liu, Yizhou, et al.
Published: (2024)

Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling
by: Trae Research Team, et al.
Published: (2025)

SR-Eval: Evaluating LLMs on Code Generation under Stepwise Requirement Refinement
by: Zhan, Zexun, et al.
Published: (2025)

A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models
by: Wu, Yixi, et al.
Published: (2024)

CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
by: Guo, Jiawei, et al.
Published: (2024)

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
by: Hu, Ruida, et al.
Published: (2024)

CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion
by: Ren, Qibing, et al.
Published: (2024)

VHDL-Eval: A Framework for Evaluating Large Language Models in VHDL Code Generation
by: Vijayaraghavan, Prashanth, et al.
Published: (2024)

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
by: Cao, Jialun, et al.
Published: (2024)

Large Language Models for Code Summarization
by: Szalontai, Balázs, et al.
Published: (2024)

Empirical Study of Code Large Language Models for Binary Security Patch Detection
by: Li, Qingyuan, et al.
Published: (2025)

CODEMENV: Benchmarking Large Language Models on Code Migration
by: Cheng, Keyuan, et al.
Published: (2025)

CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation
by: Wang, Peiding, et al.
Published: (2025)

CodeMirage: Hallucinations in Code Generated by Large Language Models
by: Agarwal, Vibhor, et al.
Published: (2024)

A Survey on Code Generation with LLM-based Agents
by: Dong, Yihong, et al.
Published: (2025)

CodeMind: Evaluating Large Language Models for Code Reasoning
by: Liu, Changshu, et al.
Published: (2024)

Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases
by: Wong, Sherman, et al.
Published: (2025)

AXIOM: Benchmarking LLM-as-a-Judge for Code via Rule-Based Perturbation and Multisource Quality Calibration
by: Wang, Ruiqi, et al.
Published: (2025)

DDPT: Diffusion-Driven Prompt Tuning for Large Language Model Code Generation
by: Li, Jinyang, et al.
Published: (2025)

A Multi-Expert Large Language Model Architecture for Verilog Code Generation
by: Nadimi, Bardia, et al.
Published: (2024)

Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions
by: Cassano, Federico, et al.
Published: (2023)

ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages
by: Kammakomati, Mehant, et al.
Published: (2024)

code_transformed: The Influence of Large Language Models on Code
by: Xu, Yuliang, et al.
Published: (2025)

ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models
by: Han, Hojae, et al.
Published: (2024)

Evaluating Quantized Large Language Models for Code Generation on Low-Resource Language Benchmarks
by: Nyamsuren, Enkhbold
Published: (2024)

RovoDev Code Reviewer: A Large-Scale Online Evaluation of LLM-based Code Review Automation at Atlassian
by: Tantithamthavorn, Kla, et al.
Published: (2026)

Is Functional Correctness Enough to Evaluate Code Language Models? Exploring Diversity of Generated Codes
by: Chon, Heejae, et al.
Published: (2024)

Investigating the Efficacy of Large Language Models for Code Clone Detection
by: Khajezade, Mohamad, et al.
Published: (2024)

A Survey on Large Language Models for Code Generation
by: Jiang, Juyong, et al.
Published: (2024)

ICE-Score: Instructing Large Language Models to Evaluate Code
by: Zhuo, Terry Yue
Published: (2023)

A Code Comprehension Benchmark for Large Language Models for Code
by: Havare, Jayant, et al.
Published: (2025)