Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.08161 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909692285943808 |
|---|---|
| author | Gao, Zuchen Zhan, Zizheng Li, Xianming Yu, Erxin Zhan, Ziqi Zhang, Haotian Chen, Bin Zhang, Yuqun Li, Jing |
| author_facet | Gao, Zuchen Zhan, Zizheng Li, Xianming Yu, Erxin Zhan, Ziqi Zhang, Haotian Chen, Bin Zhang, Yuqun Li, Jing |
| contents | Code embeddings capture the semantic representations of code and are crucial for various code-related large language model (LLM) applications, such as code search. Previous training primarily relies on optimizing the InfoNCE loss by comparing positive natural language (NL)-code pairs with in-batch negatives. However, due to the sparse nature of code contexts, training solely by comparing the major differences between positive and negative pairs may fail to capture deeper semantic nuances. To address this issue, we propose a novel order-augmented strategy for improved code search (OASIS). It leverages order-based similarity labels to train models to capture subtle differences in similarity among negative pairs. Extensive benchmark evaluations demonstrate that our OASIS model significantly outperforms previous state-of-the-art models focusing solely on major positive-negative differences. It underscores the value of exploiting subtle differences among negative pairs with order labels for effective code embedding training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08161 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OASIS: Order-Augmented Strategy for Improved Code Search Gao, Zuchen Zhan, Zizheng Li, Xianming Yu, Erxin Zhan, Ziqi Zhang, Haotian Chen, Bin Zhang, Yuqun Li, Jing Computation and Language Information Retrieval Code embeddings capture the semantic representations of code and are crucial for various code-related large language model (LLM) applications, such as code search. Previous training primarily relies on optimizing the InfoNCE loss by comparing positive natural language (NL)-code pairs with in-batch negatives. However, due to the sparse nature of code contexts, training solely by comparing the major differences between positive and negative pairs may fail to capture deeper semantic nuances. To address this issue, we propose a novel order-augmented strategy for improved code search (OASIS). It leverages order-based similarity labels to train models to capture subtle differences in similarity among negative pairs. Extensive benchmark evaluations demonstrate that our OASIS model significantly outperforms previous state-of-the-art models focusing solely on major positive-negative differences. It underscores the value of exploiting subtle differences among negative pairs with order labels for effective code embedding training. |
| title | OASIS: Order-Augmented Strategy for Improved Code Search |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2503.08161 |