K-ASTRO: Structure-Aware Adaptation of LLMs for Code Vulnerability Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917009362518016 |
|---|---|
| author | Zhang, Yifan Sandborn, Michael Larson, Stefan Huang, Yu Leach, Kevin |
| author_facet | Zhang, Yifan Sandborn, Michael Larson, Stefan Huang, Yu Leach, Kevin |
| contents | Large Language Models (LLMs) are transforming software engineering tasks, including code vulnerability detection-a critical area of software security. However, existing methods often rely on resource-intensive models or graph-based techniques, limiting their accessibility and practicality. This paper introduces K-ASTRO, a lightweight Transformer model that combines semantic embeddings from LLMs with structural features of Abstract Syntax Trees (ASTs) to improve both efficiency and accuracy in code vulnerability detection. Our approach introduces an AST-based augmentation technique inspired by mutation testing, a structure-aware attention mechanism that incorporates augmented AST features, and a joint adaptation pipeline to unify code semantics and syntax. Experimental results on three large-scale datasets, including BigVul, DiverseVul, and PrimeVul-demonstrate state-of-the-art performance while enabling rapid inference on CPUs with minimal training time. By offering a scalable, interpretable, and efficient solution, K-ASTRO bridges the gap between LLM advancements and practical software vulnerability detection, providing open-sourced tools to foster further research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2208_08067 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | K-ASTRO: Structure-Aware Adaptation of LLMs for Code Vulnerability Detection Zhang, Yifan Sandborn, Michael Larson, Stefan Huang, Yu Leach, Kevin Software Engineering Machine Learning Large Language Models (LLMs) are transforming software engineering tasks, including code vulnerability detection-a critical area of software security. However, existing methods often rely on resource-intensive models or graph-based techniques, limiting their accessibility and practicality. This paper introduces K-ASTRO, a lightweight Transformer model that combines semantic embeddings from LLMs with structural features of Abstract Syntax Trees (ASTs) to improve both efficiency and accuracy in code vulnerability detection. Our approach introduces an AST-based augmentation technique inspired by mutation testing, a structure-aware attention mechanism that incorporates augmented AST features, and a joint adaptation pipeline to unify code semantics and syntax. Experimental results on three large-scale datasets, including BigVul, DiverseVul, and PrimeVul-demonstrate state-of-the-art performance while enabling rapid inference on CPUs with minimal training time. By offering a scalable, interpretable, and efficient solution, K-ASTRO bridges the gap between LLM advancements and practical software vulnerability detection, providing open-sourced tools to foster further research. |
| title | K-ASTRO: Structure-Aware Adaptation of LLMs for Code Vulnerability Detection |
| topic | Software Engineering Machine Learning |
| url | https://arxiv.org/abs/2208.08067 |