morgan-heisler/DeckBench: v1.0.0 — DECKBench Initial Release (KDD 2026)
Fuente:
Zenodo
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Recurso digital |
| Publié: |
Zenodo
2026
|
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866902234925629440 |
|---|---|
| author | dsjang2 Morgan Heisler |
| author_facet | dsjang2 Morgan Heisler |
| contents | <h1>DECKBench v1.0.0</h1> <p>Official benchmark release accompanying the KDD 2026 paper: <strong>DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing</strong></p> <h2>What's Included</h2> <ul> <li><strong>294 paper–slide pair URLs</strong> spanning CVPR, ECCV, ICLR, and ICML across multiple years</li> <li><strong>Evaluation code</strong> for both tasks: paper-to-slide generation (Task 1) and multi-turn iterative editing (Task 2)</li> <li><strong>User simulation pipeline</strong> with three personas (Granular Analyst, Balanced Editor, Executive) for generating realistic multi-turn editing instructions</li> <li><strong>Baseline editor agent</strong> built on the OpenAI Agents framework, with an abstract base class for plug-in custom agents</li> <li><strong>Full metric suite</strong>: perplexity, faithfulness, text/figure similarity, layout quality, DTW, transition similarity, ΔDTW, ΔTransSim</li> <li><strong>Tutorial notebook</strong> for the user simulator (<code>user_simulator_tutorial.ipynb</code>)</li> <li><strong>Pre-generated baseline decks</strong> available on <a href="https://huggingface.co/datasets/mheisler/DeckBench">HuggingFace</a></li> </ul> <h2>Citation</h2> <p>If you use DECKBench in your research, please cite:</p> <pre><code>@misc{jang2026deckbench, title = {DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing}, author = {Daesik Jang and Morgan Lindsay Heisler and Linzi Xing and Yifei Li and Edward Wang and Ying Xiong and Yong Zhang and Zhenan Fan}, year = {2026}, eprint = {2602.13318}, archivePrefix = {arXiv}, primaryClass = {cs.AI}, url = {https://arxiv.org/abs/2602.13318} }</code></pre> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_20318469 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | morgan-heisler/DeckBench: v1.0.0 — DECKBench Initial Release (KDD 2026) dsjang2 Morgan Heisler <h1>DECKBench v1.0.0</h1> <p>Official benchmark release accompanying the KDD 2026 paper: <strong>DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing</strong></p> <h2>What's Included</h2> <ul> <li><strong>294 paper–slide pair URLs</strong> spanning CVPR, ECCV, ICLR, and ICML across multiple years</li> <li><strong>Evaluation code</strong> for both tasks: paper-to-slide generation (Task 1) and multi-turn iterative editing (Task 2)</li> <li><strong>User simulation pipeline</strong> with three personas (Granular Analyst, Balanced Editor, Executive) for generating realistic multi-turn editing instructions</li> <li><strong>Baseline editor agent</strong> built on the OpenAI Agents framework, with an abstract base class for plug-in custom agents</li> <li><strong>Full metric suite</strong>: perplexity, faithfulness, text/figure similarity, layout quality, DTW, transition similarity, ΔDTW, ΔTransSim</li> <li><strong>Tutorial notebook</strong> for the user simulator (<code>user_simulator_tutorial.ipynb</code>)</li> <li><strong>Pre-generated baseline decks</strong> available on <a href="https://huggingface.co/datasets/mheisler/DeckBench">HuggingFace</a></li> </ul> <h2>Citation</h2> <p>If you use DECKBench in your research, please cite:</p> <pre><code>@misc{jang2026deckbench, title = {DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing}, author = {Daesik Jang and Morgan Lindsay Heisler and Linzi Xing and Yifei Li and Edward Wang and Ying Xiong and Yong Zhang and Zhenan Fan}, year = {2026}, eprint = {2602.13318}, archivePrefix = {arXiv}, primaryClass = {cs.AI}, url = {https://arxiv.org/abs/2602.13318} }</code></pre> |
| title | morgan-heisler/DeckBench: v1.0.0 — DECKBench Initial Release (KDD 2026) |
| url | https://doi.org/10.5281/zenodo.20318469 |