CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909120893812736 |
|---|---|
| author | Wang, Hao Gao, Zeyu Zhang, Chao Sha, Zihan Sun, Mingyang Zhou, Yuchen Zhu, Wenyu Sun, Wenju Qiu, Han Xiao, Xi |
| author_facet | Wang, Hao Gao, Zeyu Zhang, Chao Sha, Zihan Sun, Mingyang Zhou, Yuchen Zhu, Wenyu Sun, Wenju Qiu, Han Xiao, Xi |
| contents | Binary code representation learning has shown significant performance in binary analysis tasks. But existing solutions often have poor transferability, particularly in few-shot and zero-shot scenarios where few or no training samples are available for the tasks. To address this problem, we present CLAP (Contrastive Language-Assembly Pre-training), which employs natural language supervision to learn better representations of binary code (i.e., assembly code) and get better transferability. At the core, our approach boosts superior transfer learning capabilities by effectively aligning binary code with their semantics explanations (in natural language), resulting a model able to generate better embeddings for binary code. To enable this alignment training, we then propose an efficient dataset engine that could automatically generate a large and diverse dataset comprising of binary code and corresponding natural language explanations. We have generated 195 million pairs of binary code and explanations and trained a prototype of CLAP. The evaluations of CLAP across various downstream tasks in binary analysis all demonstrate exceptional performance. Notably, without any task-specific training, CLAP is often competitive with a fully supervised baseline, showing excellent transferability. We release our pre-trained model and code at https://github.com/Hustcw/CLAP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_16928 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision Wang, Hao Gao, Zeyu Zhang, Chao Sha, Zihan Sun, Mingyang Zhou, Yuchen Zhu, Wenyu Sun, Wenju Qiu, Han Xiao, Xi Software Engineering Artificial Intelligence Binary code representation learning has shown significant performance in binary analysis tasks. But existing solutions often have poor transferability, particularly in few-shot and zero-shot scenarios where few or no training samples are available for the tasks. To address this problem, we present CLAP (Contrastive Language-Assembly Pre-training), which employs natural language supervision to learn better representations of binary code (i.e., assembly code) and get better transferability. At the core, our approach boosts superior transfer learning capabilities by effectively aligning binary code with their semantics explanations (in natural language), resulting a model able to generate better embeddings for binary code. To enable this alignment training, we then propose an efficient dataset engine that could automatically generate a large and diverse dataset comprising of binary code and corresponding natural language explanations. We have generated 195 million pairs of binary code and explanations and trained a prototype of CLAP. The evaluations of CLAP across various downstream tasks in binary analysis all demonstrate exceptional performance. Notably, without any task-specific training, CLAP is often competitive with a fully supervised baseline, showing excellent transferability. We release our pre-trained model and code at https://github.com/Hustcw/CLAP. |
| title | CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2402.16928 |