Integrating Large Language Models and Knowledge Graphs for Extraction and Validation of Textual Test Data
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909411215147008 |
|---|---|
| author | De Santis, Antonio Balduini, Marco De Santis, Federico Proia, Andrea Leo, Arsenio Brambilla, Marco Della Valle, Emanuele |
| author_facet | De Santis, Antonio Balduini, Marco De Santis, Federico Proia, Andrea Leo, Arsenio Brambilla, Marco Della Valle, Emanuele |
| contents | Aerospace manufacturing companies, such as Thales Alenia Space, design, develop, integrate, verify, and validate products characterized by high complexity and low volume. They carefully document all phases for each product but analyses across products are challenging due to the heterogeneity and unstructured nature of the data in documents. In this paper, we propose a hybrid methodology that leverages Knowledge Graphs (KGs) in conjunction with Large Language Models (LLMs) to extract and validate data contained in these documents. We consider a case study focused on test data related to electronic boards for satellites. To do so, we extend the Semantic Sensor Network ontology. We store the metadata of the reports in a KG, while the actual test results are stored in parquet accessible via a Virtual Knowledge Graph. The validation process is managed using an LLM-based approach. We also conduct a benchmarking study to evaluate the performance of state-of-the-art LLMs in executing this task. Finally, we analyze the costs and benefits of automating preexisting processes of manual data extraction and validation for subsequent cross-report analyses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_01700 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Integrating Large Language Models and Knowledge Graphs for Extraction and Validation of Textual Test Data De Santis, Antonio Balduini, Marco De Santis, Federico Proia, Andrea Leo, Arsenio Brambilla, Marco Della Valle, Emanuele Artificial Intelligence Computation and Language Aerospace manufacturing companies, such as Thales Alenia Space, design, develop, integrate, verify, and validate products characterized by high complexity and low volume. They carefully document all phases for each product but analyses across products are challenging due to the heterogeneity and unstructured nature of the data in documents. In this paper, we propose a hybrid methodology that leverages Knowledge Graphs (KGs) in conjunction with Large Language Models (LLMs) to extract and validate data contained in these documents. We consider a case study focused on test data related to electronic boards for satellites. To do so, we extend the Semantic Sensor Network ontology. We store the metadata of the reports in a KG, while the actual test results are stored in parquet accessible via a Virtual Knowledge Graph. The validation process is managed using an LLM-based approach. We also conduct a benchmarking study to evaluate the performance of state-of-the-art LLMs in executing this task. Finally, we analyze the costs and benefits of automating preexisting processes of manual data extraction and validation for subsequent cross-report analyses. |
| title | Integrating Large Language Models and Knowledge Graphs for Extraction and Validation of Textual Test Data |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2408.01700 |