Large Language Models for Code Analysis: Do LLMs Really Do Their Job?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909129205874688 |
|---|---|
| author | Fang, Chongzhou Miao, Ning Srivastav, Shaurya Liu, Jialin Zhang, Ruoyu Fang, Ruijie Asmita Tsang, Ryan Nazari, Najmeh Wang, Han Homayoun, Houman |
| author_facet | Fang, Chongzhou Miao, Ning Srivastav, Shaurya Liu, Jialin Zhang, Ruoyu Fang, Ruijie Asmita Tsang, Ryan Nazari, Najmeh Wang, Han Homayoun, Houman |
| contents | Large language models (LLMs) have demonstrated significant potential in the realm of natural language understanding and programming code processing tasks. Their capacity to comprehend and generate human-like code has spurred research into harnessing LLMs for code analysis purposes. However, the existing body of literature falls short in delivering a systematic evaluation and assessment of LLMs' effectiveness in code analysis, particularly in the context of obfuscated code.
This paper seeks to bridge this gap by offering a comprehensive evaluation of LLMs' capabilities in performing code analysis tasks. Additionally, it presents real-world case studies that employ LLMs for code analysis. Our findings indicate that LLMs can indeed serve as valuable tools for automating code analysis, albeit with certain limitations. Through meticulous exploration, this research contributes to a deeper understanding of the potential and constraints associated with utilizing LLMs in code analysis, paving the way for enhanced applications in this critical domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_12357 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Large Language Models for Code Analysis: Do LLMs Really Do Their Job? Fang, Chongzhou Miao, Ning Srivastav, Shaurya Liu, Jialin Zhang, Ruoyu Fang, Ruijie Asmita Tsang, Ryan Nazari, Najmeh Wang, Han Homayoun, Houman Software Engineering Cryptography and Security Large language models (LLMs) have demonstrated significant potential in the realm of natural language understanding and programming code processing tasks. Their capacity to comprehend and generate human-like code has spurred research into harnessing LLMs for code analysis purposes. However, the existing body of literature falls short in delivering a systematic evaluation and assessment of LLMs' effectiveness in code analysis, particularly in the context of obfuscated code. This paper seeks to bridge this gap by offering a comprehensive evaluation of LLMs' capabilities in performing code analysis tasks. Additionally, it presents real-world case studies that employ LLMs for code analysis. Our findings indicate that LLMs can indeed serve as valuable tools for automating code analysis, albeit with certain limitations. Through meticulous exploration, this research contributes to a deeper understanding of the potential and constraints associated with utilizing LLMs in code analysis, paving the way for enhanced applications in this critical domain. |
| title | Large Language Models for Code Analysis: Do LLMs Really Do Their Job? |
| topic | Software Engineering Cryptography and Security |
| url | https://arxiv.org/abs/2310.12357 |