SEE++: Evolving Snowpark Execution Environment for Modern Workloads
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914160164470784 |
|---|---|
| author | Jain, Gaurav Baker, Brandon Yin, Joe Xie, Chenwei Ye, Zihao Kulkarni, Sidh Abdelrahman, Sara Qi, Nova Shrestha, Urjeet Halcrow, Mike Bailey, Dave He, Yuxiong |
| author_facet | Jain, Gaurav Baker, Brandon Yin, Joe Xie, Chenwei Ye, Zihao Kulkarni, Sidh Abdelrahman, Sara Qi, Nova Shrestha, Urjeet Halcrow, Mike Bailey, Dave He, Yuxiong |
| contents | Snowpark enables Data Engineering and AI/ML workloads to run directly within Snowflake by deploying a secure sandbox on virtual warehouse nodes. This Snowpark Execution Environment (SEE) allows users to execute arbitrary workloads in Python and other languages in a secure and performant manner. As adoption has grown, the diversity of workloads has introduced increasingly sophisticated needs for sandboxing. To address these evolving requirements, Snowpark transitioned its in-house sandboxing solution to gVisor, augmented with targeted optimizations. This paper describes both the functional and performance objectives that guided the upgrade, outlines the new sandbox architecture, and details the challenges encountered during the journey, along with the solutions developed to resolve them. Finally, we present case studies that highlight new features enabled by the upgraded architecture, demonstrating SEE's extensibility and flexibility in supporting the next generation of Snowpark workloads. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_12457 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SEE++: Evolving Snowpark Execution Environment for Modern Workloads Jain, Gaurav Baker, Brandon Yin, Joe Xie, Chenwei Ye, Zihao Kulkarni, Sidh Abdelrahman, Sara Qi, Nova Shrestha, Urjeet Halcrow, Mike Bailey, Dave He, Yuxiong Databases Distributed, Parallel, and Cluster Computing Snowpark enables Data Engineering and AI/ML workloads to run directly within Snowflake by deploying a secure sandbox on virtual warehouse nodes. This Snowpark Execution Environment (SEE) allows users to execute arbitrary workloads in Python and other languages in a secure and performant manner. As adoption has grown, the diversity of workloads has introduced increasingly sophisticated needs for sandboxing. To address these evolving requirements, Snowpark transitioned its in-house sandboxing solution to gVisor, augmented with targeted optimizations. This paper describes both the functional and performance objectives that guided the upgrade, outlines the new sandbox architecture, and details the challenges encountered during the journey, along with the solutions developed to resolve them. Finally, we present case studies that highlight new features enabled by the upgraded architecture, demonstrating SEE's extensibility and flexibility in supporting the next generation of Snowpark workloads. |
| title | SEE++: Evolving Snowpark Execution Environment for Modern Workloads |
| topic | Databases Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2511.12457 |