AdvFusion: Adapter-based Knowledge Transfer for Code Summarization on Code Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909436941959168 |
|---|---|
| author | Saberi, Iman Esmaeili, Amirreza Fard, Fatemeh Chen, Fuxiang |
| author_facet | Saberi, Iman Esmaeili, Amirreza Fard, Fatemeh Chen, Fuxiang |
| contents | Programming languages can benefit from one another by utilizing a pre-trained model for software engineering tasks such as code summarization and method name prediction. While full fine-tuning of Code Language Models (Code-LMs) has been explored for multilingual knowledge transfer, research on Parameter Efficient Fine-Tuning (PEFT) for this purpose is limited. AdapterFusion, a PEFT architecture, aims to enhance task performance by leveraging information from multiple languages but primarily focuses on the target language.
To address this, we propose AdvFusion, a novel PEFT-based approach that effectively learns from other languages before adapting to the target task. Evaluated on code summarization and method name prediction, AdvFusion outperforms AdapterFusion by up to 1.7 points and surpasses LoRA with gains of 1.99, 1.26, and 2.16 for Ruby, JavaScript, and Go, respectively. We open-source our scripts for replication purposes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_07854 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | AdvFusion: Adapter-based Knowledge Transfer for Code Summarization on Code Language Models Saberi, Iman Esmaeili, Amirreza Fard, Fatemeh Chen, Fuxiang Software Engineering 68N30, 68T35 D.2.0; I.2.5 Programming languages can benefit from one another by utilizing a pre-trained model for software engineering tasks such as code summarization and method name prediction. While full fine-tuning of Code Language Models (Code-LMs) has been explored for multilingual knowledge transfer, research on Parameter Efficient Fine-Tuning (PEFT) for this purpose is limited. AdapterFusion, a PEFT architecture, aims to enhance task performance by leveraging information from multiple languages but primarily focuses on the target language. To address this, we propose AdvFusion, a novel PEFT-based approach that effectively learns from other languages before adapting to the target task. Evaluated on code summarization and method name prediction, AdvFusion outperforms AdapterFusion by up to 1.7 points and surpasses LoRA with gains of 1.99, 1.26, and 2.16 for Ruby, JavaScript, and Go, respectively. We open-source our scripts for replication purposes. |
| title | AdvFusion: Adapter-based Knowledge Transfer for Code Summarization on Code Language Models |
| topic | Software Engineering 68N30, 68T35 D.2.0; I.2.5 |
| url | https://arxiv.org/abs/2307.07854 |