Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.19757 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909623207854080 |
|---|---|
| author | Dziuba, Maria Malykh, Valentin |
| author_facet | Dziuba, Maria Malykh, Valentin |
| contents | Effective generation of structured code comments requires robust quality metrics for dataset curation, yet existing approaches (SIDE, MIDQ, STASIS) suffer from limited code-comment analysis. We propose CIDRe, a language-agnostic reference-free quality criterion combining four synergistic aspects: (1) relevance (code-comment semantic alignment), (2) informativeness (functional coverage), (3) completeness (presence of all structure sections), and (4) description length (detail sufficiency). We validate our criterion on a manually annotated dataset. Experiments demonstrate CIDRe's superiority over existing metrics, achieving improvement in cross-entropy evaluation. When applied to filter comments, the models finetuned on CIDRe-filtered data show statistically significant quality gains in GPT-4o-mini assessments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_19757 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement Dziuba, Maria Malykh, Valentin Software Engineering Artificial Intelligence Computation and Language Machine Learning Effective generation of structured code comments requires robust quality metrics for dataset curation, yet existing approaches (SIDE, MIDQ, STASIS) suffer from limited code-comment analysis. We propose CIDRe, a language-agnostic reference-free quality criterion combining four synergistic aspects: (1) relevance (code-comment semantic alignment), (2) informativeness (functional coverage), (3) completeness (presence of all structure sections), and (4) description length (detail sufficiency). We validate our criterion on a manually annotated dataset. Experiments demonstrate CIDRe's superiority over existing metrics, achieving improvement in cross-entropy evaluation. When applied to filter comments, the models finetuned on CIDRe-filtered data show statistically significant quality gains in GPT-4o-mini assessments. |
| title | CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement |
| topic | Software Engineering Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2505.19757 |