Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.16338 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914571127619584 |
|---|---|
| author | Filho, Cloter Migliorini Machado, Julia Graciela Silva, Edson Armando Scoczynski, Marcella |
| author_facet | Filho, Cloter Migliorini Machado, Julia Graciela Silva, Edson Armando Scoczynski, Marcella |
| contents | The large-scale digitization of historical archives has created a paradox: "dark data"-digital objects lacking metadata for retrieval. Manual archival description is slow and expensive, limiting discovery and reuse. We propose Vidya, a modular pipeline that orchestrates Large Language Models (LLMs) and FOSS tools to automate semantic enrichment and archival ingestion at scale. Vidya constrains generations using YAML-defined ontologies and Pydantic validation, producing deterministic, structured JSON outputs from probabilistic models. Developed at Laboratory for Digital Humanities and Innovation (LAMUHDI) of the State University of Ponta Grossa (UEPG), Vidya applies Maker principles and open-source practices to enable low-cost deployment in memory institutions using modest hardware. We compare LLM performance and present a cost-benefit analysis showing major gains, reducing processing time from decades to days while complying with NOBRADE and ISAD(G). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_16338 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Vidya: An AI-Driven Modular Pipeline for Archival Automation and Semantic Metadata Enrichment Filho, Cloter Migliorini Machado, Julia Graciela Silva, Edson Armando Scoczynski, Marcella Digital Libraries Computation and Language The large-scale digitization of historical archives has created a paradox: "dark data"-digital objects lacking metadata for retrieval. Manual archival description is slow and expensive, limiting discovery and reuse. We propose Vidya, a modular pipeline that orchestrates Large Language Models (LLMs) and FOSS tools to automate semantic enrichment and archival ingestion at scale. Vidya constrains generations using YAML-defined ontologies and Pydantic validation, producing deterministic, structured JSON outputs from probabilistic models. Developed at Laboratory for Digital Humanities and Innovation (LAMUHDI) of the State University of Ponta Grossa (UEPG), Vidya applies Maker principles and open-source practices to enable low-cost deployment in memory institutions using modest hardware. We compare LLM performance and present a cost-benefit analysis showing major gains, reducing processing time from decades to days while complying with NOBRADE and ISAD(G). |
| title | Vidya: An AI-Driven Modular Pipeline for Archival Automation and Semantic Metadata Enrichment |
| topic | Digital Libraries Computation and Language |
| url | https://arxiv.org/abs/2605.16338 |