Vital: Vulnerability-Oriented Symbolic Execution via Type-Unsafe Pointer-Guided Monte Carlo Tree Search

Fuente: arXiv
Enregistré dans:
Détails bibliographiques
Auteurs principaux: Tu, Haoxin, Jiang, Lingxiao, Böhme, Marcel
Format: Preprint
Publié: 2024
Sujets:
Accès en ligne:
Tags: Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
_version_ 1866915671224352768
author Tu, Haoxin
Jiang, Lingxiao
Böhme, Marcel
author_facet Tu, Haoxin
Jiang, Lingxiao
Böhme, Marcel
contents How to find memory safety bugs efficiently when navigating a symbolic execution tree that suffers from path explosion? Existing solutions either adopt path search heuristics to maximize coverage rate or chopped symbolic execution to skip uninteresting code (i.e., manually labeled as vulnerability-unrelated) during path exploration. However, most existing search heuristics are not vulnerability-oriented, and manual labeling of irrelevant code-to-be-skipped relies heavily on prior expert knowledge, making it hard to detect vulnerabilities effectively in practice. This paper proposes Vital, a new vulnerability-oriented path exploration for symbolic execution with two innovations. First, a new indicator (i.e., type-unsafe pointers) is suggested to approximate vulnerable paths. A pointer that is type-unsafe cannot be statically proven to be safely dereferenced without memory corruption. Our key hypothesis is that a path with more type-unsafe pointers is more likely to be vulnerable. Second, a new type-unsafe pointer-guided Monte Carlo Tree Search algorithm is implemented to guide the path exploration towards the areas that contain more unsafe pointers, aiming to increase the likelihood of detecting vulnerabilities. We built Vital on top of KLEE and compared it with existing path searching strategies and chopped symbolic execution. In the former, the results demonstrate that Vital could cover up to 90.03% more unsafe pointers and detect up to 57.14% more unique memory errors. In the latter, the results show that Vital could achieve a speedup of up to 30x execution time and a reduction of up to 20x memory consumption to detect known vulnerabilities without prior expert knowledge automatically. In practice, Vital also detected one previously unknown vulnerability (a new CVE ID is assigned), which has been fixed by developers.
format Preprint
id arxiv_https___arxiv_org_abs_2408_08772
institution arXiv
publishDate 2024
record_format arxiv
spellingShingle Vital: Vulnerability-Oriented Symbolic Execution via Type-Unsafe Pointer-Guided Monte Carlo Tree Search
Tu, Haoxin
Jiang, Lingxiao
Böhme, Marcel
Software Engineering
Cryptography and Security
How to find memory safety bugs efficiently when navigating a symbolic execution tree that suffers from path explosion? Existing solutions either adopt path search heuristics to maximize coverage rate or chopped symbolic execution to skip uninteresting code (i.e., manually labeled as vulnerability-unrelated) during path exploration. However, most existing search heuristics are not vulnerability-oriented, and manual labeling of irrelevant code-to-be-skipped relies heavily on prior expert knowledge, making it hard to detect vulnerabilities effectively in practice. This paper proposes Vital, a new vulnerability-oriented path exploration for symbolic execution with two innovations. First, a new indicator (i.e., type-unsafe pointers) is suggested to approximate vulnerable paths. A pointer that is type-unsafe cannot be statically proven to be safely dereferenced without memory corruption. Our key hypothesis is that a path with more type-unsafe pointers is more likely to be vulnerable. Second, a new type-unsafe pointer-guided Monte Carlo Tree Search algorithm is implemented to guide the path exploration towards the areas that contain more unsafe pointers, aiming to increase the likelihood of detecting vulnerabilities. We built Vital on top of KLEE and compared it with existing path searching strategies and chopped symbolic execution. In the former, the results demonstrate that Vital could cover up to 90.03% more unsafe pointers and detect up to 57.14% more unique memory errors. In the latter, the results show that Vital could achieve a speedup of up to 30x execution time and a reduction of up to 20x memory consumption to detect known vulnerabilities without prior expert knowledge automatically. In practice, Vital also detected one previously unknown vulnerability (a new CVE ID is assigned), which has been fixed by developers.
title Vital: Vulnerability-Oriented Symbolic Execution via Type-Unsafe Pointer-Guided Monte Carlo Tree Search
topic Software Engineering
Cryptography and Security
url https://arxiv.org/abs/2408.08772