How Fragile is Relation Extraction under Entity Replacements?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910436924850176 |
|---|---|
| author | Wang, Yiwei Hooi, Bryan Wang, Fei Cai, Yujun Liang, Yuxuan Zhou, Wenxuan Tang, Jing Duan, Manjuan Chen, Muhao |
| author_facet | Wang, Yiwei Hooi, Bryan Wang, Fei Cai, Yujun Liang, Yuxuan Zhou, Wenxuan Tang, Jing Duan, Manjuan Chen, Muhao |
| contents | Relation extraction (RE) aims to extract the relations between entity names from the textual context. In principle, textual context determines the ground-truth relation and the RE models should be able to correctly identify the relations reflected by the textual context. However, existing work has found that the RE models memorize the entity name patterns to make RE predictions while ignoring the textual context. This motivates us to raise the question: ``are RE models robust to the entity replacements?'' In this work, we operate the random and type-constrained entity replacements over the RE instances in TACRED and evaluate the state-of-the-art RE models under the entity replacements. We observe the 30\% - 50\% F1 score drops on the state-of-the-art RE models under entity replacements. These results suggest that we need more efforts to develop effective RE models robust to entity replacements. We release the source code at https://github.com/wangywUST/RobustRE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_13551 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | How Fragile is Relation Extraction under Entity Replacements? Wang, Yiwei Hooi, Bryan Wang, Fei Cai, Yujun Liang, Yuxuan Zhou, Wenxuan Tang, Jing Duan, Manjuan Chen, Muhao Computation and Language Artificial Intelligence Relation extraction (RE) aims to extract the relations between entity names from the textual context. In principle, textual context determines the ground-truth relation and the RE models should be able to correctly identify the relations reflected by the textual context. However, existing work has found that the RE models memorize the entity name patterns to make RE predictions while ignoring the textual context. This motivates us to raise the question: ``are RE models robust to the entity replacements?'' In this work, we operate the random and type-constrained entity replacements over the RE instances in TACRED and evaluate the state-of-the-art RE models under the entity replacements. We observe the 30\% - 50\% F1 score drops on the state-of-the-art RE models under entity replacements. These results suggest that we need more efforts to develop effective RE models robust to entity replacements. We release the source code at https://github.com/wangywUST/RobustRE. |
| title | How Fragile is Relation Extraction under Entity Replacements? |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2305.13551 |