Static Detection of Core Structures in Tigress Virtualization-Based Obfuscation Using an LLVM Pass
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914274575646720 |
|---|---|
| author | An, Sangjun Lee, Seoksu Cho, Eun-Sun |
| author_facet | An, Sangjun Lee, Seoksu Cho, Eun-Sun |
| contents | Malware often uses obfuscation to hinder security analysis. Among these techniques, virtualization-based obfuscation is particularly strong because it protects programs by translating original instructions into attacker-defined virtual machine (VM) bytecode, producing long and complex code that is difficult to analyze and deobfuscate. This paper aims to identify the structural components of virtualization-based obfuscation through static analysis. By examining the execution model of obfuscated code, we define and detect the key elements required for deobfuscation-namely the dispatch routine, handler blocks, and the VM region-using LLVM IR. Experimental results show that, in the absence of compiler optimizations, the proposed LLVM Pass successfully detects all core structures across major virtualization options, including switch, direct, and indirect modes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_12916 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Static Detection of Core Structures in Tigress Virtualization-Based Obfuscation Using an LLVM Pass An, Sangjun Lee, Seoksu Cho, Eun-Sun Cryptography and Security Malware often uses obfuscation to hinder security analysis. Among these techniques, virtualization-based obfuscation is particularly strong because it protects programs by translating original instructions into attacker-defined virtual machine (VM) bytecode, producing long and complex code that is difficult to analyze and deobfuscate. This paper aims to identify the structural components of virtualization-based obfuscation through static analysis. By examining the execution model of obfuscated code, we define and detect the key elements required for deobfuscation-namely the dispatch routine, handler blocks, and the VM region-using LLVM IR. Experimental results show that, in the absence of compiler optimizations, the proposed LLVM Pass successfully detects all core structures across major virtualization options, including switch, direct, and indirect modes. |
| title | Static Detection of Core Structures in Tigress Virtualization-Based Obfuscation Using an LLVM Pass |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2601.12916 |