Robust Evidence for Declining Disruptiveness: Assessing the Role of Zero-Backward-Citation Works
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910060347654144 |
|---|---|
| author | Park, Michael Leahey, Erin Funk, Russell J. |
| author_facet | Park, Michael Leahey, Erin Funk, Russell J. |
| contents | We respond to Holst et al.'s critique that the decline in scientific disruptiveness documented in Park et al. (Nature, 2023) is an artifact of including works with zero backward citations. Using their advocated dataset, metric, and exclusion criteria, we find declines equivalent to major benchmark transformations in science. Their own regression model--designed to address their concerns about zero-citation works--yields large and significant declines for both papers and patents (p<0.001), a result found in their supplementary tables yet left unaddressed, despite directly contradicting their central claim. Their critique is further undermined by severe quality issues in their data, which contain three times more zero-citation works than ours. We trace this excess to their inclusion of at least 2.8 million editorials, obituaries, and comments, 1.5 million books and proceedings, and 254,000 product and artistic reviews--in all, 20% of their sample is non-research content that almost by definition lacks backward citations. Simple keyword searches confirm the problem's severity, identifying among others 456 For Dummies guides, 50 Dr. Seuss and Curious George books, and the Captain Underpants series--all zero-citation entries in their sample. Applying granular document type classification to their data reveals that such non-research content fell from 40% to 8% of their sample between 1945 and 2010--a shift sufficient to generate the decline in zero-citation prevalence they attribute to metadata errors in our study. Standard practice excludes such content to guard against the metadata quality concerns at the center of their critique--concerns their dataset exemplifies rather than addresses. Declining disruptiveness has been documented in nearly 100 studies across multiple databases, metrics, and non-citation-based measures. The weight of evidence does not support an artifact-based explanation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_00184 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Robust Evidence for Declining Disruptiveness: Assessing the Role of Zero-Backward-Citation Works Park, Michael Leahey, Erin Funk, Russell J. Social and Information Networks Digital Libraries We respond to Holst et al.'s critique that the decline in scientific disruptiveness documented in Park et al. (Nature, 2023) is an artifact of including works with zero backward citations. Using their advocated dataset, metric, and exclusion criteria, we find declines equivalent to major benchmark transformations in science. Their own regression model--designed to address their concerns about zero-citation works--yields large and significant declines for both papers and patents (p<0.001), a result found in their supplementary tables yet left unaddressed, despite directly contradicting their central claim. Their critique is further undermined by severe quality issues in their data, which contain three times more zero-citation works than ours. We trace this excess to their inclusion of at least 2.8 million editorials, obituaries, and comments, 1.5 million books and proceedings, and 254,000 product and artistic reviews--in all, 20% of their sample is non-research content that almost by definition lacks backward citations. Simple keyword searches confirm the problem's severity, identifying among others 456 For Dummies guides, 50 Dr. Seuss and Curious George books, and the Captain Underpants series--all zero-citation entries in their sample. Applying granular document type classification to their data reveals that such non-research content fell from 40% to 8% of their sample between 1945 and 2010--a shift sufficient to generate the decline in zero-citation prevalence they attribute to metadata errors in our study. Standard practice excludes such content to guard against the metadata quality concerns at the center of their critique--concerns their dataset exemplifies rather than addresses. Declining disruptiveness has been documented in nearly 100 studies across multiple databases, metrics, and non-citation-based measures. The weight of evidence does not support an artifact-based explanation. |
| title | Robust Evidence for Declining Disruptiveness: Assessing the Role of Zero-Backward-Citation Works |
| topic | Social and Information Networks Digital Libraries |
| url | https://arxiv.org/abs/2503.00184 |