Large-scale Evaluation of Notebook Checkpointing with AI Agents
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910901665267712 |
|---|---|
| author | Fang, Hanxi Chockchowwat, Supawit Sundaram, Hari Park, Yongjoo |
| author_facet | Fang, Hanxi Chockchowwat, Supawit Sundaram, Hari Park, Yongjoo |
| contents | Saving, or checkpointing, intermediate results during interactive data exploration can potentially boost user productivity. However, existing studies on this topic are limited, as they primarily rely on small-scale experiments with human participants - a fundamental constraint of human subject studies. To address this limitation, we employ AI agents to simulate a large number of complex data exploration scenarios, including revisiting past states and branching into new exploration paths. This strategy enables us to accurately assess the impact of checkpointing while closely mimicking the behavior of real-world data practitioners. Our evaluation results, involving more than 1,000 exploration paths and 2,848 executed code blocks, show that a checkpointing framework for computational notebooks can indeed enhance productivity by minimizing unnecessary code re-executions and redundant variables or code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_01377 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Large-scale Evaluation of Notebook Checkpointing with AI Agents Fang, Hanxi Chockchowwat, Supawit Sundaram, Hari Park, Yongjoo Human-Computer Interaction Saving, or checkpointing, intermediate results during interactive data exploration can potentially boost user productivity. However, existing studies on this topic are limited, as they primarily rely on small-scale experiments with human participants - a fundamental constraint of human subject studies. To address this limitation, we employ AI agents to simulate a large number of complex data exploration scenarios, including revisiting past states and branching into new exploration paths. This strategy enables us to accurately assess the impact of checkpointing while closely mimicking the behavior of real-world data practitioners. Our evaluation results, involving more than 1,000 exploration paths and 2,848 executed code blocks, show that a checkpointing framework for computational notebooks can indeed enhance productivity by minimizing unnecessary code re-executions and redundant variables or code. |
| title | Large-scale Evaluation of Notebook Checkpointing with AI Agents |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2504.01377 |