Dynamic Retrieval Augmented Generation of Ontologies using Artificial Intelligence (DRAGON-AI)
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917808948903936 |
|---|---|
| author | Toro, Sabrina Anagnostopoulos, Anna V Bello, Sue Blumberg, Kai Cameron, Rhiannon Carmody, Leigh Diehl, Alexander D Dooley, Damion Duncan, William Fey, Petra Gaudet, Pascale Harris, Nomi L Joachimiak, Marcin Kiani, Leila Lubiana, Tiago Munoz-Torres, Monica C O'Neil, Shawn Osumi-Sutherland, David Puig, Aleix Reese, Justin P Reiser, Leonore Robb, Sofia Ruemping, Troy Seager, James Sid, Eric Stefancsik, Ray Weber, Magalie Wood, Valerie Haendel, Melissa A Mungall, Christopher J |
| author_facet | Toro, Sabrina Anagnostopoulos, Anna V Bello, Sue Blumberg, Kai Cameron, Rhiannon Carmody, Leigh Diehl, Alexander D Dooley, Damion Duncan, William Fey, Petra Gaudet, Pascale Harris, Nomi L Joachimiak, Marcin Kiani, Leila Lubiana, Tiago Munoz-Torres, Monica C O'Neil, Shawn Osumi-Sutherland, David Puig, Aleix Reese, Justin P Reiser, Leonore Robb, Sofia Ruemping, Troy Seager, James Sid, Eric Stefancsik, Ray Weber, Magalie Wood, Valerie Haendel, Melissa A Mungall, Christopher J |
| contents | Background: Ontologies are fundamental components of informatics infrastructure in domains such as biomedical, environmental, and food sciences, representing consensus knowledge in an accurate and computable form. However, their construction and maintenance demand substantial resources and necessitate substantial collaboration between domain experts, curators, and ontology experts. We present Dynamic Retrieval Augmented Generation of Ontologies using AI (DRAGON-AI), an ontology generation method employing Large Language Models (LLMs) and Retrieval Augmented Generation (RAG). DRAGON-AI can generate textual and logical ontology components, drawing from existing knowledge in multiple ontologies and unstructured text sources.
Results: We assessed performance of DRAGON-AI on de novo term construction across ten diverse ontologies, making use of extensive manual evaluation of results. Our method has high precision for relationship generation, but has slightly lower precision than from logic-based reasoning. Our method is also able to generate definitions deemed acceptable by expert evaluators, but these scored worse than human-authored definitions. Notably, evaluators with the highest level of confidence in a domain were better able to discern flaws in AI-generated definitions. We also demonstrated the ability of DRAGON-AI to incorporate natural language instructions in the form of GitHub issues.
Conclusions: These findings suggest DRAGON-AI's potential to substantially aid the manual ontology construction process. However, our results also underscore the importance of having expert curators and ontology editors drive the ontology generation process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_10904 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Dynamic Retrieval Augmented Generation of Ontologies using Artificial Intelligence (DRAGON-AI) Toro, Sabrina Anagnostopoulos, Anna V Bello, Sue Blumberg, Kai Cameron, Rhiannon Carmody, Leigh Diehl, Alexander D Dooley, Damion Duncan, William Fey, Petra Gaudet, Pascale Harris, Nomi L Joachimiak, Marcin Kiani, Leila Lubiana, Tiago Munoz-Torres, Monica C O'Neil, Shawn Osumi-Sutherland, David Puig, Aleix Reese, Justin P Reiser, Leonore Robb, Sofia Ruemping, Troy Seager, James Sid, Eric Stefancsik, Ray Weber, Magalie Wood, Valerie Haendel, Melissa A Mungall, Christopher J Artificial Intelligence Background: Ontologies are fundamental components of informatics infrastructure in domains such as biomedical, environmental, and food sciences, representing consensus knowledge in an accurate and computable form. However, their construction and maintenance demand substantial resources and necessitate substantial collaboration between domain experts, curators, and ontology experts. We present Dynamic Retrieval Augmented Generation of Ontologies using AI (DRAGON-AI), an ontology generation method employing Large Language Models (LLMs) and Retrieval Augmented Generation (RAG). DRAGON-AI can generate textual and logical ontology components, drawing from existing knowledge in multiple ontologies and unstructured text sources. Results: We assessed performance of DRAGON-AI on de novo term construction across ten diverse ontologies, making use of extensive manual evaluation of results. Our method has high precision for relationship generation, but has slightly lower precision than from logic-based reasoning. Our method is also able to generate definitions deemed acceptable by expert evaluators, but these scored worse than human-authored definitions. Notably, evaluators with the highest level of confidence in a domain were better able to discern flaws in AI-generated definitions. We also demonstrated the ability of DRAGON-AI to incorporate natural language instructions in the form of GitHub issues. Conclusions: These findings suggest DRAGON-AI's potential to substantially aid the manual ontology construction process. However, our results also underscore the importance of having expert curators and ontology editors drive the ontology generation process. |
| title | Dynamic Retrieval Augmented Generation of Ontologies using Artificial Intelligence (DRAGON-AI) |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2312.10904 |