CausalCite: A Causal Formulation of Paper Citations
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916262501679104 |
|---|---|
| author | Kumar, Ishan Jin, Zhijing Mokhtarian, Ehsan Guo, Siyuan Chen, Yuen Sachan, Mrinmaya Schölkopf, Bernhard |
| author_facet | Kumar, Ishan Jin, Zhijing Mokhtarian, Ehsan Guo, Siyuan Chen, Yuen Sachan, Mrinmaya Schölkopf, Bernhard |
| contents | Citation count of a paper is a commonly used proxy for evaluating the significance of a paper in the scientific community. Yet citation measures are widely criticized for failing to accurately reflect the true impact of a paper. Thus, we propose CausalCite, a new way to measure the significance of a paper by assessing the causal impact of the paper on its follow-up papers. CausalCite is based on a novel causal inference method, TextMatch, which adapts the traditional matching framework to high-dimensional text embeddings. TextMatch encodes each paper using text embeddings from large language models (LLMs), extracts similar samples by cosine similarity, and synthesizes a counterfactual sample as the weighted average of similar papers according to their similarity values. We demonstrate the effectiveness of CausalCite on various criteria, such as high correlation with paper impact as reported by scientific experts on a previous dataset of 1K papers, (test-of-time) awards for past papers, and its stability across various subfields of AI. We also provide a set of findings that can serve as suggested ways for future researchers to use our metric for a better understanding of the quality of a paper. Our code is available at https://github.com/causalNLP/causal-cite. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_02790 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CausalCite: A Causal Formulation of Paper Citations Kumar, Ishan Jin, Zhijing Mokhtarian, Ehsan Guo, Siyuan Chen, Yuen Sachan, Mrinmaya Schölkopf, Bernhard Computation and Language Artificial Intelligence Computers and Society Information Retrieval Machine Learning Citation count of a paper is a commonly used proxy for evaluating the significance of a paper in the scientific community. Yet citation measures are widely criticized for failing to accurately reflect the true impact of a paper. Thus, we propose CausalCite, a new way to measure the significance of a paper by assessing the causal impact of the paper on its follow-up papers. CausalCite is based on a novel causal inference method, TextMatch, which adapts the traditional matching framework to high-dimensional text embeddings. TextMatch encodes each paper using text embeddings from large language models (LLMs), extracts similar samples by cosine similarity, and synthesizes a counterfactual sample as the weighted average of similar papers according to their similarity values. We demonstrate the effectiveness of CausalCite on various criteria, such as high correlation with paper impact as reported by scientific experts on a previous dataset of 1K papers, (test-of-time) awards for past papers, and its stability across various subfields of AI. We also provide a set of findings that can serve as suggested ways for future researchers to use our metric for a better understanding of the quality of a paper. Our code is available at https://github.com/causalNLP/causal-cite. |
| title | CausalCite: A Causal Formulation of Paper Citations |
| topic | Computation and Language Artificial Intelligence Computers and Society Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2311.02790 |