CodeCloak: A Method for Evaluating and Mitigating Code Leakage by LLM Code Assistants
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916458594828288 |
|---|---|
| author | Noah, Amit Finkman Shapira, Avishag Kochva, Eden Bar Maimon, Inbar Mimran, Dudu Elovici, Yuval Shabtai, Asaf |
| author_facet | Noah, Amit Finkman Shapira, Avishag Kochva, Eden Bar Maimon, Inbar Mimran, Dudu Elovici, Yuval Shabtai, Asaf |
| contents | LLM-based code assistants are becoming increasingly popular among developers. These tools help developers improve their coding efficiency and reduce errors by providing real-time suggestions based on the developer's codebase. While beneficial, the use of these tools can inadvertently expose the developer's proprietary code to the code assistant service provider during the development process. In this work, we propose a method to mitigate the risk of code leakage when using LLM-based code assistants. CodeCloak is a novel deep reinforcement learning agent that manipulates the prompts before sending them to the code assistant service. CodeCloak aims to achieve the following two contradictory goals: (i) minimizing code leakage, while (ii) preserving relevant and useful suggestions for the developer. Our evaluation, employing StarCoder and Code Llama, LLM-based code assistants models, demonstrates CodeCloak's effectiveness on a diverse set of code repositories of varying sizes, as well as its transferability across different models. We also designed a method for reconstructing the developer's original codebase from code segments sent to the code assistant service (i.e., prompts) during the development process, to thoroughly analyze code leakage risks and evaluate the effectiveness of CodeCloak under practical development scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_09066 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CodeCloak: A Method for Evaluating and Mitigating Code Leakage by LLM Code Assistants Noah, Amit Finkman Shapira, Avishag Kochva, Eden Bar Maimon, Inbar Mimran, Dudu Elovici, Yuval Shabtai, Asaf Cryptography and Security Computation and Language Machine Learning Programming Languages LLM-based code assistants are becoming increasingly popular among developers. These tools help developers improve their coding efficiency and reduce errors by providing real-time suggestions based on the developer's codebase. While beneficial, the use of these tools can inadvertently expose the developer's proprietary code to the code assistant service provider during the development process. In this work, we propose a method to mitigate the risk of code leakage when using LLM-based code assistants. CodeCloak is a novel deep reinforcement learning agent that manipulates the prompts before sending them to the code assistant service. CodeCloak aims to achieve the following two contradictory goals: (i) minimizing code leakage, while (ii) preserving relevant and useful suggestions for the developer. Our evaluation, employing StarCoder and Code Llama, LLM-based code assistants models, demonstrates CodeCloak's effectiveness on a diverse set of code repositories of varying sizes, as well as its transferability across different models. We also designed a method for reconstructing the developer's original codebase from code segments sent to the code assistant service (i.e., prompts) during the development process, to thoroughly analyze code leakage risks and evaluate the effectiveness of CodeCloak under practical development scenarios. |
| title | CodeCloak: A Method for Evaluating and Mitigating Code Leakage by LLM Code Assistants |
| topic | Cryptography and Security Computation and Language Machine Learning Programming Languages |
| url | https://arxiv.org/abs/2404.09066 |