Analyzing the Effectiveness of Listwise Reranking with Positional Invariance on Temporal Generalizability
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912051103793152 |
|---|---|
| author | Yoon, Soyoung Kim, Jongyoon Hwang, Seung-won |
| author_facet | Yoon, Soyoung Kim, Jongyoon Hwang, Seung-won |
| contents | This working note outlines our participation in the retrieval task at CLEF 2024. We highlight the considerable gap between studying retrieval performance on static knowledge documents and understanding performance in real-world environments. Therefore, Addressing these discrepancies and measuring the temporal persistence of IR systems is crucial. By investigating the LongEval benchmark, specifically designed for such dynamic environments, our findings demonstrate the effectiveness of a listwise reranking approach, which proficiently handles inaccuracies induced by temporal distribution shifts. Among listwise rerankers, our findings show that ListT5, which effectively mitigates the positional bias problem by adopting the Fusion-in-Decoder architecture, is especially effective, and more so, as temporal drift increases, on the test-long subset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_06716 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Analyzing the Effectiveness of Listwise Reranking with Positional Invariance on Temporal Generalizability Yoon, Soyoung Kim, Jongyoon Hwang, Seung-won Information Retrieval This working note outlines our participation in the retrieval task at CLEF 2024. We highlight the considerable gap between studying retrieval performance on static knowledge documents and understanding performance in real-world environments. Therefore, Addressing these discrepancies and measuring the temporal persistence of IR systems is crucial. By investigating the LongEval benchmark, specifically designed for such dynamic environments, our findings demonstrate the effectiveness of a listwise reranking approach, which proficiently handles inaccuracies induced by temporal distribution shifts. Among listwise rerankers, our findings show that ListT5, which effectively mitigates the positional bias problem by adopting the Fusion-in-Decoder architecture, is especially effective, and more so, as temporal drift increases, on the test-long subset. |
| title | Analyzing the Effectiveness of Listwise Reranking with Positional Invariance on Temporal Generalizability |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2407.06716 |