Cover Image

DeepCRCEval: Revisiting the Evaluation of Code Review Comment Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lu, Junyi, Li, Xiaojia, Hua, Zihan, Yu, Lei, Cheng, Shiqi, Yang, Li, Zhang, Fengjun, Zuo, Chun
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Artificial Intelligence Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Practical Defect-Focused Automated Code Review
by: Lu, Junyi, et al.
Published: (2025)

MOS: Towards Effective Smart Contract Vulnerability Detection through Mixture-of-Experts Tuning of Large Language Models
by: Yuan, Hang, et al.
Published: (2025)

CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells
by: Naik, Atharva, et al.
Published: (2024)

Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection
by: Yu, Lei, et al.
Published: (2025)

SQL-Commenter: Aligning Large Language Models for SQL Comment Generation with Direct Preference Optimization
by: Yu, Lei, et al.
Published: (2026)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

Code-Vision: Evaluating Multimodal LLMs Logic Understanding and Code Generation Capabilities
by: Wang, Hanbin, et al.
Published: (2025)

CodeContests+: High-Quality Test Case Generation for Competitive Programming
by: Wang, Zihan, et al.
Published: (2025)

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks
by: Zhao, Songwen, et al.
Published: (2025)

From If-Statements to ML Pipelines: Revisiting Bias in Code-Generation
by: Bui, Minh Duc, et al.
Published: (2026)

Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance
by: Song, Yewei, et al.
Published: (2024)

AI-Mediated Code Comment Improvement
by: Dhakal, Maria, et al.
Published: (2025)

FairCoder: Evaluating Social Bias of LLMs in Code Generation
by: Du, Yongkang, et al.
Published: (2025)

Evaluation of Code LLMs on Geospatial Code Generation
by: Gramacki, Piotr, et al.
Published: (2024)

AI-Assisted Fixes to Code Review Comments at Scale
by: Maddila, Chandra, et al.
Published: (2025)

Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generation
by: Haider, Md. Asif, et al.
Published: (2024)

SAEL: Leveraging Large Language Models with Adaptive Mixture-of-Experts for Smart Contract Vulnerability Detection
by: Yu, Lei, et al.
Published: (2025)

FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
by: Li, Wei, et al.
Published: (2025)

Python Symbolic Execution with LLM-powered Code Generation
by: Wang, Wenhan, et al.
Published: (2024)

Evaluating and Achieving Controllable Code Completion in Code LLM
by: Zhang, Jiajun, et al.
Published: (2026)

ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation
by: Zhang, Chenchen, et al.
Published: (2025)

Zero-shot Evaluation of Deep Learning for Java Code Clone Detection
by: Heinze, Thomas S.
Published: (2026)

Retrieval-Augmented Code Review Comment Generation
by: Hong, Hyunsun, et al.
Published: (2025)

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval
by: Wang, Jiexin, et al.
Published: (2024)

Code Fingerprints: Disentangled Attribution of LLM-Generated Code
by: Guo, Jiaxun, et al.
Published: (2026)

Showing LLM-Generated Code Selectively Based on Confidence of LLMs
by: Li, Jia, et al.
Published: (2024)

CodeFlowBench: A Multi-turn, Iterative Benchmark for Complex Code Generation
by: Wang, Sizhe, et al.
Published: (2025)

ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation
by: Liu, Kaiyuan, et al.
Published: (2025)

Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey
by: Wang, Junqiao, et al.
Published: (2024)

StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning
by: Wang, Hao, et al.
Published: (2026)

AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions
by: Wang, Zihan, et al.
Published: (2025)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation
by: Wang, Peiding, et al.
Published: (2025)

FeatBench: Towards More Realistic Evaluation of Feature-level Code Generation
by: Chen, Haorui, et al.
Published: (2025)

HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
by: Yu, Zhaojian, et al.
Published: (2024)

VersiCode: Towards Version-controllable Code Generation
by: Wu, Tongtong, et al.
Published: (2024)

Functional Consistency of LLM Code Embeddings: A Self-Evolving Data Synthesis Framework for Benchmarking
by: Li, Zhuohao, et al.
Published: (2025)

Enhancing Code Generation via Bidirectional Comment-Level Mutual Grounding
by: Di, Yifeng, et al.
Published: (2025)

CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models
by: Lin, Hong Yi, et al.
Published: (2025)

MaintainCoder: Maintainable Code Generation Under Dynamic Requirements
by: Wang, Zhengren, et al.
Published: (2025)