Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.20081 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909660148137984 |
|---|---|
| author | Gupta, Dhruv Lakshmy, Gayathri Ganesh Xie, Yiqing |
| author_facet | Gupta, Dhruv Lakshmy, Gayathri Ganesh Xie, Yiqing |
| contents | Retrieval-Augmented Code Generation (RACG) is a critical technique for enhancing code generation by retrieving relevant information. In this work, we conduct an in-depth analysis of code retrieval by systematically masking specific features while preserving code functionality. Our discoveries include: (1) although trained on code, current retrievers heavily rely on surface-level textual features (e.g., docstrings, identifier names), and (2) they exhibit a strong bias towards well-documented code, even if the documentation is irrelevant. Based on our discoveries, we propose SACL, a framework that enriches textual information and reduces bias by augmenting code or structural knowledge with semantic information. Extensive experiments show that SACL substantially improves code retrieval (e.g., by 12.8% / 9.4% / 7.0% Recall@1 on HumanEval / MBPP / SWE-Bench-Lite), which also leads to better code generation performance (e.g., by 4.88% Pass@1 on HumanEval). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20081 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SACL: Understanding and Combating Textual Bias in Code Retrieval with Semantic-Augmented Reranking and Localization Gupta, Dhruv Lakshmy, Gayathri Ganesh Xie, Yiqing Computation and Language Artificial Intelligence Retrieval-Augmented Code Generation (RACG) is a critical technique for enhancing code generation by retrieving relevant information. In this work, we conduct an in-depth analysis of code retrieval by systematically masking specific features while preserving code functionality. Our discoveries include: (1) although trained on code, current retrievers heavily rely on surface-level textual features (e.g., docstrings, identifier names), and (2) they exhibit a strong bias towards well-documented code, even if the documentation is irrelevant. Based on our discoveries, we propose SACL, a framework that enriches textual information and reduces bias by augmenting code or structural knowledge with semantic information. Extensive experiments show that SACL substantially improves code retrieval (e.g., by 12.8% / 9.4% / 7.0% Recall@1 on HumanEval / MBPP / SWE-Bench-Lite), which also leads to better code generation performance (e.g., by 4.88% Pass@1 on HumanEval). |
| title | SACL: Understanding and Combating Textual Bias in Code Retrieval with Semantic-Augmented Reranking and Localization |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.20081 |