Do we still need canaries in the coal mine? Measuring shadow stack effectiveness in countering stack smashing

Fuente: arXiv
Saved in:
Bibliographic Details
Main Authors: Depuydt, Hugo, Gülmez, Merve, Nyman, Thomas, Mühlberg, Jan Tobias
Format: Preprint
Published: 2024
Subjects:
Online Access:
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1866915074936930304
author Depuydt, Hugo
Gülmez, Merve
Nyman, Thomas
Mühlberg, Jan Tobias
author_facet Depuydt, Hugo
Gülmez, Merve
Nyman, Thomas
Mühlberg, Jan Tobias
contents Stack canaries and shadow stacks are widely deployed mitigations to memory-safety vulnerabilities. While stack canaries are introduced by the compiler and rely on sentry values placed between variables and control data, shadow stack implementations protect return addresses explicitly and rely on hardware features available in modern processor designs for efficiency. In this paper we hypothesize that stack canaries and shadow stacks provide similar levels of protections against sequential stack-based overflows. Based on the Juliet test suite, we evaluate whether 64-bit x86 (x86-64) systems benefit from enabling stack canaries in addition to the x86-64 shadow stack enforcement. We observe divergence in overflow detection rates between the GCC and Clang compilers and across optimization levels, which we attribute to differences in stack layouts generated by the compilers. We also find that x86-64 shadow stack implementations are more effective and outperform stack canaries when combined with a stack-protector-like stack layout. We implement and evaluate an enhancement to the Clang x86-64 shadow stack instrumentation that improves the shadow stack detection accuracy based on this observation.
format Preprint
id arxiv_https___arxiv_org_abs_2412_16343
institution arXiv
publishDate 2024
record_format arxiv
spellingShingle Do we still need canaries in the coal mine? Measuring shadow stack effectiveness in countering stack smashing
Depuydt, Hugo
Gülmez, Merve
Nyman, Thomas
Mühlberg, Jan Tobias
Cryptography and Security
Stack canaries and shadow stacks are widely deployed mitigations to memory-safety vulnerabilities. While stack canaries are introduced by the compiler and rely on sentry values placed between variables and control data, shadow stack implementations protect return addresses explicitly and rely on hardware features available in modern processor designs for efficiency. In this paper we hypothesize that stack canaries and shadow stacks provide similar levels of protections against sequential stack-based overflows. Based on the Juliet test suite, we evaluate whether 64-bit x86 (x86-64) systems benefit from enabling stack canaries in addition to the x86-64 shadow stack enforcement. We observe divergence in overflow detection rates between the GCC and Clang compilers and across optimization levels, which we attribute to differences in stack layouts generated by the compilers. We also find that x86-64 shadow stack implementations are more effective and outperform stack canaries when combined with a stack-protector-like stack layout. We implement and evaluate an enhancement to the Clang x86-64 shadow stack instrumentation that improves the shadow stack detection accuracy based on this observation.
title Do we still need canaries in the coal mine? Measuring shadow stack effectiveness in countering stack smashing
topic Cryptography and Security
url https://arxiv.org/abs/2412.16343