Cover Image

CodeFuse-CommitEval: Towards Benchmarking LLM's Power on Commit Message and Code Change Inconsistency Detection

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Qingyu, Liu, Puzhuo, Di, Peng, Qian, Chenxiong
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model
by: Di, Peng, et al.
Published: (2023)

Analyzing Message-Code Inconsistency in AI Coding Agent-Authored Pull Requests
by: Gong, Jingzhi, et al.
Published: (2026)

Code Researcher: Deep Research Agent for Large Systems Code and Commit History
by: Singh, Ramneet, et al.
Published: (2025)

Rover: Context-aware Conflict Resolution with LLM
by: Zhang, Qingyu, et al.
Published: (2026)

KADEL: Knowledge-Aware Denoising Learning for Commit Message Generation
by: Tao, Wei, et al.
Published: (2024)

AI-Powered Commit Explorer (APCE)
by: Grees, Yousab, et al.
Published: (2025)

CodeFuse-CR-Bench: A Comprehensiveness-aware Benchmark for End-to-End Code Review Evaluation in Python Projects
by: Guo, Hanyang, et al.
Published: (2025)

Can LLM Generate Regression Tests for Software Commits?
by: Liu, Jing, et al.
Published: (2025)

AI-Augmented CI/CD Pipelines: From Code Commit to Production with Autonomous Decisions
by: Baqar, Mohammad, et al.
Published: (2025)

LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation
by: Kuhar, Sachit, et al.
Published: (2024)

Lore: Repurposing Git Commit Messages as a Structured Knowledge Protocol for AI Coding Agents
by: Stetsenko, Ivan
Published: (2026)

Brevity is the Soul of Wit: Condensing Code Changes to Improve Commit Message Generation
by: Kuang, Hongyu, et al.
Published: (2025)

Contextual Code Retrieval for Commit Message Generation: A Preliminary Study
by: Xiong, Bo, et al.
Published: (2025)

GitHub's Copilot Code Review: Can AI Spot Security Flaws Before You Commit?
by: Amro, Amena, et al.
Published: (2025)

COMET: Generating Commit Messages using Delta Graph Context Representation
by: Mandli, Abhinav Reddy, et al.
Published: (2024)

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
by: Paul, Debalina Ghosh, et al.
Published: (2024)

REPOFUSE: Repository-Level Code Completion with Fused Dual Context
by: Liang, Ming, et al.
Published: (2024)

An Empirical Analysis of Git Commit Logs for Potential Inconsistency in Code Clones
by: Yokomori, Reishi, et al.
Published: (2024)

CommitSuite: A Comprehensive Benchmark for Commit Classification and Message Generation
by: Wan, Zirui, et al.
Published: (2026)

CommitBench: A Benchmark for Commit Message Generation
by: Schall, Maximilian, et al.
Published: (2024)

LLM-Driven Collaborative Model for Untangling Commits via Explicit and Implicit Dependency Reasoning
by: Hou, Bo, et al.
Published: (2025)

CodeFuse-Query: A Data-Centric Static Code Analysis System for Large-Scale Organizations
by: Xie, Xiaoheng, et al.
Published: (2024)

LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery
by: Akhavan, Arshia, et al.
Published: (2025)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
by: Zhang, Tanghaoran, et al.
Published: (2026)

Commit0: Library Generation from Scratch
by: Zhao, Wenting, et al.
Published: (2024)

Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models
by: Wang, Yanlin, et al.
Published: (2024)

RepoMasterEval: Evaluating Code Completion via Real-World Repositories
by: Wu, Qinyun, et al.
Published: (2024)

RedCode: Risky Code Execution and Generation Benchmark for Code Agents
by: Guo, Chengquan, et al.
Published: (2024)

Specification and Detection of LLM Code Smells
by: Mahmoudi, Brahim, et al.
Published: (2025)

RAG-Enhanced Commit Message Generation
by: Zhang, Linghao, et al.
Published: (2024)

RustEvo^2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation
by: Liang, Linxi, et al.
Published: (2025)

CodeCircuit: Toward Inferring LLM-Generated Code Correctness via Attribution Graphs
by: He, Yicheng, et al.
Published: (2026)

Code Copycat Conundrum: Demystifying Repetition in LLM-based Code Generation
by: Liu, Mingwei, et al.
Published: (2025)

CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning
by: Roy, Monoshi Kumar, et al.
Published: (2025)

CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
by: Zheng, Qinkai, et al.
Published: (2023)

EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems
by: Zhang, Wentao, et al.
Published: (2026)

Re-Evaluating Code LLM Benchmarks Under Semantic Mutation
by: Pan, Zhiyuan, et al.
Published: (2025)

CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding
by: Luo, Hanjun, et al.
Published: (2025)

Detecting Multiple Semantic Concerns in Tangled Code Commits
by: Koh, Beomsu, et al.
Published: (2026)

Conventional Commit Classification using Large Language Models and Prompt Engineering
by: Quadir, H. M. Sazzad, et al.
Published: (2026)