Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2510.12023 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908591382855680 |
|---|---|
| author | Kwak, Alice Saebom Alexeeva, Maria Hahn-Powell, Gus Alcock, Keith McLaughlin, Kevin McCorkle, Doug McNunn, Gabe Surdeanu, Mihai |
| author_facet | Kwak, Alice Saebom Alexeeva, Maria Hahn-Powell, Gus Alcock, Keith McLaughlin, Kevin McCorkle, Doug McNunn, Gabe Surdeanu, Mihai |
| contents | The current trend in information extraction (IE) is to rely extensively on large language models, effectively discarding decades of experience in building symbolic or statistical IE systems. This paper compares a neuro-symbolic (NS) and an LLM-based IE system in the agricultural domain, evaluating them on nine interviews across pork, dairy, and crop subdomains. The LLM-based system outperforms the NS one (F1 total: 69.4 vs. 52.7; core: 63.0 vs. 47.2), where total includes all extracted information and core focuses on essential details. However, each system has trade-offs: the NS approach offers faster runtime, greater control, and high accuracy in context-free tasks but lacks generalizability, struggles with contextual nuances, and requires significant resources to develop and maintain. The LLM-based system achieves higher performance, faster deployment, and easier maintenance but has slower runtime, limited control, model dependency and hallucination risks. Our findings highlight the "hidden cost" of deploying NLP systems in real-world applications, emphasizing the need to balance performance, efficiency, and control. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_12023 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Information Extraction from Conversation Transcripts: Neuro-Symbolic vs. LLM Kwak, Alice Saebom Alexeeva, Maria Hahn-Powell, Gus Alcock, Keith McLaughlin, Kevin McCorkle, Doug McNunn, Gabe Surdeanu, Mihai Computation and Language The current trend in information extraction (IE) is to rely extensively on large language models, effectively discarding decades of experience in building symbolic or statistical IE systems. This paper compares a neuro-symbolic (NS) and an LLM-based IE system in the agricultural domain, evaluating them on nine interviews across pork, dairy, and crop subdomains. The LLM-based system outperforms the NS one (F1 total: 69.4 vs. 52.7; core: 63.0 vs. 47.2), where total includes all extracted information and core focuses on essential details. However, each system has trade-offs: the NS approach offers faster runtime, greater control, and high accuracy in context-free tasks but lacks generalizability, struggles with contextual nuances, and requires significant resources to develop and maintain. The LLM-based system achieves higher performance, faster deployment, and easier maintenance but has slower runtime, limited control, model dependency and hallucination risks. Our findings highlight the "hidden cost" of deploying NLP systems in real-world applications, emphasizing the need to balance performance, efficiency, and control. |
| title | Information Extraction from Conversation Transcripts: Neuro-Symbolic vs. LLM |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.12023 |