Cover Image

Evaluating Generated Commit Messages with Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Qunhong, Zhang, Yuxia, Ma, Zexiong, Jiang, Bo, Sun, Ningyuan, Stol, Klaas-Jan, Mou, Xingyu, Liu, Hui
Format:	Preprint
Published:	2025
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Teaching Theorizing in Software Engineering Research
by: Stol, Klaas-Jan
Published: (2024)

How Are Paid and Volunteer Open Source Developers Different? A Study of the Rust Project
by: Zhang, Yuxia, et al.
Published: (2024)

Commit Messages in the Age of Large Language Models
by: Lopes, Cristina V., et al.
Published: (2024)

Tool-integrated Reinforcement Learning for Repo Deep Search
by: Ma, Zexiong, et al.
Published: (2025)

Automated Commit Message Generation with Large Language Models: An Empirical Study and Beyond
by: Xue, Pengyu, et al.
Published: (2024)

Using Large Language Models for Commit Message Generation: A Preliminary Study
by: Zhang, Linghao, et al.
Published: (2024)

Optimization is Better than Generation: Optimizing Commit Message Leveraging Human-written Commit Message
by: Li, Jiawei, et al.
Published: (2025)

RAG-Enhanced Commit Message Generation
by: Zhang, Linghao, et al.
Published: (2024)

Contextual Code Retrieval for Commit Message Generation: A Preliminary Study
by: Xiong, Bo, et al.
Published: (2025)

CommitSuite: A Comprehensive Benchmark for Commit Classification and Message Generation
by: Wan, Zirui, et al.
Published: (2026)

Brevity is the Soul of Wit: Condensing Code Changes to Improve Commit Message Generation
by: Kuang, Hongyu, et al.
Published: (2025)

Improving the Quality of Commit Messages in Students' Projects
by: Ma, Iris, et al.
Published: (2023)

Automated Generation of Commit Messages in Software Repositories
by: Palakodeti, Varun Kumar, et al.
Published: (2025)

CoRaCMG: Contextual Retrieval-Augmented Framework for Commit Message Generation
by: Xiong, Bo, et al.
Published: (2025)

CommitBench: A Benchmark for Commit Message Generation
by: Schall, Maximilian, et al.
Published: (2024)

An Empirical Study on Commit Message Generation using LLMs via In-Context Learning
by: Wu, Yifan, et al.
Published: (2025)

On the Informativeness of Security Commit Messages: A Large-scale Replication Study
by: Islam, Syful, et al.
Published: (2026)

LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLM
by: Zhang, Yuxin, et al.
Published: (2025)

Context Conquers Parameters: Outperforming Proprietary LLM in Commit Message Generation
by: Imani, Aaron, et al.
Published: (2024)

CoMRAT: Commit Message Rationale Analysis Tool
by: Dhaouadi, Mouna, et al.
Published: (2025)

Do I Belong? Modeling Sense of Virtual Community Among Linux Kernel Contributors
by: Trinkenreich, Bianca, et al.
Published: (2023)

KADEL: Knowledge-Aware Denoising Learning for Commit Message Generation
by: Tao, Wei, et al.
Published: (2024)

An Empirical Study on the Potential of LLMs in Automated Software Refactoring
by: Liu, Bo, et al.
Published: (2024)

Assertion Messages with Large Language Models (LLMs) for Code
by: Aljohani, Ahmed, et al.
Published: (2025)

On the Evaluation of Large Language Models in Unit Test Generation
by: Yang, Lin, et al.
Published: (2024)

Rationale Dataset and Analysis for the Commit Messages of the Linux Kernel Out-of-Memory Killer
by: Dhaouadi, Mouna, et al.
Published: (2024)

On the Use of Commit Messages for Corrective Software Maintenance: A Systematic Mapping Study
by: Islam, Syful, et al.
Published: (2026)

CodeFuse-CommitEval: Towards Benchmarking LLM's Power on Commit Message and Code Change Inconsistency Detection
by: Zhang, Qingyu, et al.
Published: (2025)

Compositional API Recommendation for Library-Oriented Code Generation
by: Ma, Zexiong, et al.
Published: (2024)

A Model for Understanding and Reducing Developer Burnout
by: Trinkenreich, Bianca, et al.
Published: (2023)

Empirical Evaluation of Large Language Models in Automated Program Repair
by: Sun, Jiajun, et al.
Published: (2025)

Towards Realistic Evaluation of Commit Message Generation by Matching Online and Offline Settings
by: Tsvetkov, Petr, et al.
Published: (2024)

Consider What Humans Consider: Optimizing Commit Message Leveraging Contexts Considered By Human
by: Li, Jiawei, et al.
Published: (2025)

Wired for Reuse: Automating Context-Aware Code Adaptation in IDEs via LLM-Based Agent
by: Wang, Taiming, et al.
Published: (2025)

A Survey on Evaluating Large Language Models in Code Generation Tasks
by: Chen, Liguo, et al.
Published: (2024)

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
by: He, Mengliang, et al.
Published: (2025)

An Empirical Study on LLM-based Agents for Automated Bug Fixing
by: Meng, Xiangxin, et al.
Published: (2024)

COMET: Generating Commit Messages using Delta Graph Context Representation
by: Mandli, Abhinav Reddy, et al.
Published: (2024)

From Evaluation to Enhancement: Large Language Models for Zero-Knowledge Proof Code Generation
by: Xue, Zhantong, et al.
Published: (2025)

KCoEvo: A Knowledge Graph Augmented Framework for Evolutionary Code Generation
by: Kang, Jiazhen, et al.
Published: (2026)