Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.01369 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909561870352384 |
|---|---|
| author | Zhang, Lin Gu, Zhouhong Zheng, Suhang Wang, Tao Li, Tianyu Feng, Hongwei Xiao, Yanghua |
| author_facet | Zhang, Lin Gu, Zhouhong Zheng, Suhang Wang, Tao Li, Tianyu Feng, Hongwei Xiao, Yanghua |
| contents | This paper presents LITE, an LLM-based evaluation method designed for efficient and flexible assessment of taxonomy quality. To address challenges in large-scale taxonomy evaluation, such as efficiency, fairness, and consistency, LITE adopts a top-down hierarchical evaluation strategy, breaking down the taxonomy into manageable substructures and ensuring result reliability through cross-validation and standardized input formats. LITE also introduces a penalty mechanism to handle extreme cases and provides both quantitative performance analysis and qualitative insights by integrating evaluation metrics closely aligned with task objectives. Experimental results show that LITE demonstrates high reliability in complex evaluation tasks, effectively identifying semantic errors, logical contradictions, and structural flaws in taxonomies, while offering directions for improvement. Code is available at https://github.com/Zhang-l-i-n/TAXONOMY_DETECT . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_01369 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LITE: LLM-Impelled efficient Taxonomy Evaluation Zhang, Lin Gu, Zhouhong Zheng, Suhang Wang, Tao Li, Tianyu Feng, Hongwei Xiao, Yanghua Computation and Language Information Retrieval This paper presents LITE, an LLM-based evaluation method designed for efficient and flexible assessment of taxonomy quality. To address challenges in large-scale taxonomy evaluation, such as efficiency, fairness, and consistency, LITE adopts a top-down hierarchical evaluation strategy, breaking down the taxonomy into manageable substructures and ensuring result reliability through cross-validation and standardized input formats. LITE also introduces a penalty mechanism to handle extreme cases and provides both quantitative performance analysis and qualitative insights by integrating evaluation metrics closely aligned with task objectives. Experimental results show that LITE demonstrates high reliability in complex evaluation tasks, effectively identifying semantic errors, logical contradictions, and structural flaws in taxonomies, while offering directions for improvement. Code is available at https://github.com/Zhang-l-i-n/TAXONOMY_DETECT . |
| title | LITE: LLM-Impelled efficient Taxonomy Evaluation |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2504.01369 |