Assessing the quality of information extraction
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917673380610048 |
|---|---|
| author | Seitl, Filip Kovářík, Tomáš Mirshahi, Soheyla Kryštůfek, Jan Dujava, Rastislav Ondreička, Matúš Ullrich, Herbert Gronat, Petr |
| author_facet | Seitl, Filip Kovářík, Tomáš Mirshahi, Soheyla Kryštůfek, Jan Dujava, Rastislav Ondreička, Matúš Ullrich, Herbert Gronat, Petr |
| contents | Advances in large language models have notably enhanced the efficiency of information extraction from unstructured and semi-structured data sources. As these technologies become integral to various applications, establishing an objective measure for the quality of information extraction becomes imperative. However, the scarcity of labeled data presents significant challenges to this endeavor. In this paper, we introduce an automatic framework to assess the quality of the information extraction/retrieval and its completeness. The framework focuses on information extraction in the form of entity and its properties. We discuss how to handle the input/output size limitations of the large language models and analyze their performance when extracting the information. In particular, we introduce scores to evaluate the quality of the extraction and provide an extensive discussion on how to interpret them. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_04068 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Assessing the quality of information extraction Seitl, Filip Kovářík, Tomáš Mirshahi, Soheyla Kryštůfek, Jan Dujava, Rastislav Ondreička, Matúš Ullrich, Herbert Gronat, Petr Computation and Language Advances in large language models have notably enhanced the efficiency of information extraction from unstructured and semi-structured data sources. As these technologies become integral to various applications, establishing an objective measure for the quality of information extraction becomes imperative. However, the scarcity of labeled data presents significant challenges to this endeavor. In this paper, we introduce an automatic framework to assess the quality of the information extraction/retrieval and its completeness. The framework focuses on information extraction in the form of entity and its properties. We discuss how to handle the input/output size limitations of the large language models and analyze their performance when extracting the information. In particular, we introduce scores to evaluate the quality of the extraction and provide an extensive discussion on how to interpret them. |
| title | Assessing the quality of information extraction |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.04068 |