Transforming Expert Knowledge into Scalable Ontology via Large Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913889043611648 |
|---|---|
| author | Itoku, Ikkei Theil, David Uehara, Evelyn Eichelsdoerfer Bhaduri, Sreyoshi Kuroda, Junnosuke Yumoto, Toshi Gil, Alex Perez, Natalie Cherukuri, Rajesh Nayyar, Naumaan |
| author_facet | Itoku, Ikkei Theil, David Uehara, Evelyn Eichelsdoerfer Bhaduri, Sreyoshi Kuroda, Junnosuke Yumoto, Toshi Gil, Alex Perez, Natalie Cherukuri, Rajesh Nayyar, Naumaan |
| contents | Having a unified, coherent taxonomy is essential for effective knowledge representation in domain-specific applications as diverse terminologies need to be mapped to underlying concepts. Traditional manual approaches to taxonomy alignment rely on expert review of concept pairs, but this becomes prohibitively expensive and time-consuming at scale, while subjective interpretations often lead to expert disagreements. Existing automated methods for taxonomy alignment have shown promise but face limitations in handling nuanced semantic relationships and maintaining consistency across different domains. These approaches often struggle with context-dependent concept mappings and lack transparent reasoning processes. We propose a novel framework that combines large language models (LLMs) with expert calibration and iterative prompt optimization to automate taxonomy alignment. Our method integrates expert-labeled examples, multi-stage prompt engineering, and human validation to guide LLMs in generating both taxonomy linkages and supporting rationales. In evaluating our framework on a domain-specific mapping task of concept essentiality, we achieved an F1-score of 0.97, substantially exceeding the human benchmark of 0.68. These results demonstrate the effectiveness of our approach in scaling taxonomy alignment while maintaining high-quality mappings and preserving expert oversight for ambiguous cases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_08422 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Transforming Expert Knowledge into Scalable Ontology via Large Language Models Itoku, Ikkei Theil, David Uehara, Evelyn Eichelsdoerfer Bhaduri, Sreyoshi Kuroda, Junnosuke Yumoto, Toshi Gil, Alex Perez, Natalie Cherukuri, Rajesh Nayyar, Naumaan Artificial Intelligence Having a unified, coherent taxonomy is essential for effective knowledge representation in domain-specific applications as diverse terminologies need to be mapped to underlying concepts. Traditional manual approaches to taxonomy alignment rely on expert review of concept pairs, but this becomes prohibitively expensive and time-consuming at scale, while subjective interpretations often lead to expert disagreements. Existing automated methods for taxonomy alignment have shown promise but face limitations in handling nuanced semantic relationships and maintaining consistency across different domains. These approaches often struggle with context-dependent concept mappings and lack transparent reasoning processes. We propose a novel framework that combines large language models (LLMs) with expert calibration and iterative prompt optimization to automate taxonomy alignment. Our method integrates expert-labeled examples, multi-stage prompt engineering, and human validation to guide LLMs in generating both taxonomy linkages and supporting rationales. In evaluating our framework on a domain-specific mapping task of concept essentiality, we achieved an F1-score of 0.97, substantially exceeding the human benchmark of 0.68. These results demonstrate the effectiveness of our approach in scaling taxonomy alignment while maintaining high-quality mappings and preserving expert oversight for ambiguous cases. |
| title | Transforming Expert Knowledge into Scalable Ontology via Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.08422 |