Do Code LLMs Understand Design Patterns?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916558330134528 |
|---|---|
| author | Pan, Zhenyu Song, Xuefeng Wang, Yunkun Cao, Rongyu Li, Binhua Li, Yongbin Liu, Han |
| author_facet | Pan, Zhenyu Song, Xuefeng Wang, Yunkun Cao, Rongyu Li, Binhua Li, Yongbin Liu, Han |
| contents | Code Large Language Models (LLMs) demonstrate great versatility in adapting to various downstream tasks, including code generation and completion, as well as bug detection and fixing. However, Code LLMs often fail to capture existing coding standards, leading to the generation of code that conflicts with the required design patterns for a given project. As a result, developers must post-process to adapt the generated code to the project's design norms. In this work, we empirically investigate the biases of Code LLMs in software development. Through carefully designed experiments, we assess the models' understanding of design patterns across recognition, comprehension, and generation. Our findings reveal that biases in Code LLMs significantly affect the reliability of downstream tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_04835 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do Code LLMs Understand Design Patterns? Pan, Zhenyu Song, Xuefeng Wang, Yunkun Cao, Rongyu Li, Binhua Li, Yongbin Liu, Han Software Engineering Artificial Intelligence Code Large Language Models (LLMs) demonstrate great versatility in adapting to various downstream tasks, including code generation and completion, as well as bug detection and fixing. However, Code LLMs often fail to capture existing coding standards, leading to the generation of code that conflicts with the required design patterns for a given project. As a result, developers must post-process to adapt the generated code to the project's design norms. In this work, we empirically investigate the biases of Code LLMs in software development. Through carefully designed experiments, we assess the models' understanding of design patterns across recognition, comprehension, and generation. Our findings reveal that biases in Code LLMs significantly affect the reliability of downstream tasks. |
| title | Do Code LLMs Understand Design Patterns? |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2501.04835 |