Reassessing Java Code Readability Models with a Human-Centered Approach
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911765315452928 |
|---|---|
| author | Sergeyuk, Agnia Lvova, Olga Titov, Sergey Serova, Anastasiia Bagirov, Farid Kirillova, Evgeniia Bryksin, Timofey |
| author_facet | Sergeyuk, Agnia Lvova, Olga Titov, Sergey Serova, Anastasiia Bagirov, Farid Kirillova, Evgeniia Bryksin, Timofey |
| contents | To ensure that Large Language Models (LLMs) effectively support user productivity, they need to be adjusted. Existing Code Readability (CR) models can guide this alignment. However, there are concerns about their relevance in modern software engineering since they often miss the developers' notion of readability and rely on outdated code. This research assesses existing Java CR models for LLM adjustments, measuring the correlation between their and developers' evaluations of AI-generated Java code. Using the Repertory Grid Technique with 15 developers, we identified 12 key code aspects influencing CR that were consequently assessed by 390 programmers when labeling 120 AI-generated snippets. Our findings indicate that when AI generates concise and executable code, it is often considered readable by CR models and developers. However, a limited correlation between these evaluations underscores the importance of future research on learning objectives for adjusting LLMs and on the aspects influencing CR evaluations included in predictive models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_14936 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Reassessing Java Code Readability Models with a Human-Centered Approach Sergeyuk, Agnia Lvova, Olga Titov, Sergey Serova, Anastasiia Bagirov, Farid Kirillova, Evgeniia Bryksin, Timofey Software Engineering Human-Computer Interaction To ensure that Large Language Models (LLMs) effectively support user productivity, they need to be adjusted. Existing Code Readability (CR) models can guide this alignment. However, there are concerns about their relevance in modern software engineering since they often miss the developers' notion of readability and rely on outdated code. This research assesses existing Java CR models for LLM adjustments, measuring the correlation between their and developers' evaluations of AI-generated Java code. Using the Repertory Grid Technique with 15 developers, we identified 12 key code aspects influencing CR that were consequently assessed by 390 programmers when labeling 120 AI-generated snippets. Our findings indicate that when AI generates concise and executable code, it is often considered readable by CR models and developers. However, a limited correlation between these evaluations underscores the importance of future research on learning objectives for adjusting LLMs and on the aspects influencing CR evaluations included in predictive models. |
| title | Reassessing Java Code Readability Models with a Human-Centered Approach |
| topic | Software Engineering Human-Computer Interaction |
| url | https://arxiv.org/abs/2401.14936 |