All Entities are Not Created Equal: Examining the Long Tail for Ultra-Fine Entity Typing
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914506739810304 |
|---|---|
| author | Deshmukh, Advait Umadi, Ashwin Srinivas, Dananjay Pacheco, Maria Leonor |
| author_facet | Deshmukh, Advait Umadi, Ashwin Srinivas, Dananjay Pacheco, Maria Leonor |
| contents | Due to their capacity to acquire world knowledge from large corpora, pre-trained language models (PLMs) are extensively used in ultra-fine entity typing tasks where the space of labels is extremely large. In this work, we explore the limitations of the knowledge acquired by PLMs by proposing a novel heuristic to approximate the pre-training distribution of entities when the pre-training data is unknown. Then, we systematically demonstrate that entity-typing approaches that rely solely on the parametric knowledge of PLMs struggle significantly with entities at the long tail of the pre-training distribution, and that knowledge-infused approaches can account for some of these shortcomings. Our findings suggest that we need to go beyond PLMs to produce solutions that perform well for infrequent entities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_17355 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | All Entities are Not Created Equal: Examining the Long Tail for Ultra-Fine Entity Typing Deshmukh, Advait Umadi, Ashwin Srinivas, Dananjay Pacheco, Maria Leonor Computation and Language Due to their capacity to acquire world knowledge from large corpora, pre-trained language models (PLMs) are extensively used in ultra-fine entity typing tasks where the space of labels is extremely large. In this work, we explore the limitations of the knowledge acquired by PLMs by proposing a novel heuristic to approximate the pre-training distribution of entities when the pre-training data is unknown. Then, we systematically demonstrate that entity-typing approaches that rely solely on the parametric knowledge of PLMs struggle significantly with entities at the long tail of the pre-training distribution, and that knowledge-infused approaches can account for some of these shortcomings. Our findings suggest that we need to go beyond PLMs to produce solutions that perform well for infrequent entities. |
| title | All Entities are Not Created Equal: Examining the Long Tail for Ultra-Fine Entity Typing |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.17355 |