Metadata Management for AI-Augmented Data Workflows
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908483578757120 |
|---|---|
| author | Zhao, Jinjin Krishnan, Sanjay |
| author_facet | Zhao, Jinjin Krishnan, Sanjay |
| contents | AI-augmented data workflows introduce complex governance challenges, as both human and model-driven processes generate, transform, and consume data artifacts. These workflows blend heterogeneous tools, dynamic execution patterns, and opaque model decisions, making comprehensive metadata capture difficult. In this work, we present TableVault, a metadata governance framework designed for human-AI collaborative data creation. TableVault records ingestion events, traces operation status, links execution parameters to their data origins, and exposes a standardized metadata layer. By combining database-inspired guarantees with AI-oriented design, such as declarative operation builders and lineage-aware references, TableVault supports transparency and reproducibility across mixed human-model pipelines. Through a document classification case study, we demonstrate how TableVault preserves detailed lineage and operational context, enabling robust metadata management, even in partially observable execution environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_06814 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Metadata Management for AI-Augmented Data Workflows Zhao, Jinjin Krishnan, Sanjay Databases AI-augmented data workflows introduce complex governance challenges, as both human and model-driven processes generate, transform, and consume data artifacts. These workflows blend heterogeneous tools, dynamic execution patterns, and opaque model decisions, making comprehensive metadata capture difficult. In this work, we present TableVault, a metadata governance framework designed for human-AI collaborative data creation. TableVault records ingestion events, traces operation status, links execution parameters to their data origins, and exposes a standardized metadata layer. By combining database-inspired guarantees with AI-oriented design, such as declarative operation builders and lineage-aware references, TableVault supports transparency and reproducibility across mixed human-model pipelines. Through a document classification case study, we demonstrate how TableVault preserves detailed lineage and operational context, enabling robust metadata management, even in partially observable execution environments. |
| title | Metadata Management for AI-Augmented Data Workflows |
| topic | Databases |
| url | https://arxiv.org/abs/2508.06814 |