Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909044040531968 |
|---|---|
| author | Attrach, Rafi Al Fani, Rajna Lobentanzer, Sebastian Giner-Miguelez, Joan Das, Debanshu K., Varuni H. Sarwar, Nobin Ghosh, Rajat Archit, Anwai Motghare, Surbhi Parry, Christina Conrad Oala, Luis Grosso, Lara Vanschoren, Joaquin Vogler, Steffen Goswami, Sujata Rosenthal, Eric S. Ghassemi, Marzyeh McDermott, Matthew Pollard, Tom |
| author_facet | Attrach, Rafi Al Fani, Rajna Lobentanzer, Sebastian Giner-Miguelez, Joan Das, Debanshu K., Varuni H. Sarwar, Nobin Ghosh, Rajat Archit, Anwai Motghare, Surbhi Parry, Christina Conrad Oala, Luis Grosso, Lara Vanschoren, Joaquin Vogler, Steffen Goswami, Sujata Rosenthal, Eric S. Ghassemi, Marzyeh McDermott, Matthew Pollard, Tom |
| contents | Croissant has emerged as the metadata standard for machine learning datasets, providing a structured, JSON-LD-based format that makes dataset discovery, automated ingestion, and reproducible analysis machine-checkable across ML platforms. Adoption has accelerated, and NeurIPS now requires Croissant metadata in every submission to its dataset tracks. Yet in practice Croissant generation usually starts with uploading data to a public platform, a path infeasible for governed and large local repositories that hold much of the high-value data ML increasingly relies on. We release Croissant Baker, a local-first, open-source command-line tool that generates validated Croissant metadata directly from a dataset directory through a modular handler registry. We evaluate Croissant Baker on over 140 datasets, scaling to MIMIC-IV at 886 million rows and 374 Parquet files. On held-out comparisons against producer-authored or standards-derived ground truth, Croissant Baker reaches 97-100% agreement across multiple domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_15079 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets Attrach, Rafi Al Fani, Rajna Lobentanzer, Sebastian Giner-Miguelez, Joan Das, Debanshu K., Varuni H. Sarwar, Nobin Ghosh, Rajat Archit, Anwai Motghare, Surbhi Parry, Christina Conrad Oala, Luis Grosso, Lara Vanschoren, Joaquin Vogler, Steffen Goswami, Sujata Rosenthal, Eric S. Ghassemi, Marzyeh McDermott, Matthew Pollard, Tom Machine Learning Databases Digital Libraries Information Retrieval Croissant has emerged as the metadata standard for machine learning datasets, providing a structured, JSON-LD-based format that makes dataset discovery, automated ingestion, and reproducible analysis machine-checkable across ML platforms. Adoption has accelerated, and NeurIPS now requires Croissant metadata in every submission to its dataset tracks. Yet in practice Croissant generation usually starts with uploading data to a public platform, a path infeasible for governed and large local repositories that hold much of the high-value data ML increasingly relies on. We release Croissant Baker, a local-first, open-source command-line tool that generates validated Croissant metadata directly from a dataset directory through a modular handler registry. We evaluate Croissant Baker on over 140 datasets, scaling to MIMIC-IV at 886 million rows and 374 Parquet files. On held-out comparisons against producer-authored or standards-derived ground truth, Croissant Baker reaches 97-100% agreement across multiple domains. |
| title | Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets |
| topic | Machine Learning Databases Digital Libraries Information Retrieval |
| url | https://arxiv.org/abs/2605.15079 |