Free Access to World News: Reconstructing Full-Text Articles from GDELT
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912867583787008 |
|---|---|
| author | Colladon, A. Fronzetti Vestrelli, R. |
| author_facet | Colladon, A. Fronzetti Vestrelli, R. |
| contents | News data have become essential resources across various disciplines. Still, access to full-text news corpora remains challenging due to high costs and the limited availability of free alternatives. This paper presents a novel Python package (gdeltnews) that reconstructs full-text newspaper articles at near-zero cost by leveraging the Global Database of Events, Language, and Tone (GDELT) Web News NGrams 3.0 dataset. Our method merges overlapping n-grams extracted from global online news to rebuild complete articles. We validate the approach on a benchmark set of 2211 articles from major U.S. news outlets, achieving up to 95% text similarity against original articles based on Levenshtein and SequenceMatcher metrics. Our tool facilitates economic forecasting, computational social science, information science, and natural language processing applications by enabling free and large-scale access to full-text news data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_16063 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Free Access to World News: Reconstructing Full-Text Articles from GDELT Colladon, A. Fronzetti Vestrelli, R. Computation and Language Databases Information Retrieval I.2.7; H.2.8; H.3.1 News data have become essential resources across various disciplines. Still, access to full-text news corpora remains challenging due to high costs and the limited availability of free alternatives. This paper presents a novel Python package (gdeltnews) that reconstructs full-text newspaper articles at near-zero cost by leveraging the Global Database of Events, Language, and Tone (GDELT) Web News NGrams 3.0 dataset. Our method merges overlapping n-grams extracted from global online news to rebuild complete articles. We validate the approach on a benchmark set of 2211 articles from major U.S. news outlets, achieving up to 95% text similarity against original articles based on Levenshtein and SequenceMatcher metrics. Our tool facilitates economic forecasting, computational social science, information science, and natural language processing applications by enabling free and large-scale access to full-text news data. |
| title | Free Access to World News: Reconstructing Full-Text Articles from GDELT |
| topic | Computation and Language Databases Information Retrieval I.2.7; H.2.8; H.3.1 |
| url | https://arxiv.org/abs/2504.16063 |