CCF Database (Apache Parquet mirror): column-oriented edition of the Canadian Climate Framing corpus (1978–2024)
Fuente:
Zenodo
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Recurso digital |
| Language: | English |
| Published: |
Zenodo
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866902256575578112 |
|---|---|
| author | Lemor, Antoine Pillod, Alizée Taylor, Matthew Nadeau, Richard |
| author_facet | Lemor, Antoine Pillod, Alizée Taylor, Matthew Nadeau, Richard |
| contents | <p>The Canadian Climate Framing (CCF) Database is a comprehensive, machine-learning-annotated corpus of climate-change media coverage in Canada. It comprises 266,271 articles from 20 major Canadian newspapers (1978–2024) processed into 9,198,958 two-sentence analytical units (82.9 % English, 17.1 % French). Each unit is annotated across 65 hierarchical categories by 128 BERT and CamemBERT classifiers, with a macro F1 of 0.866 on a 1,000-sentence gold standard (blind-phase Cohen's κ = 0.596, Krippendorff's α = 0.698, Gwet's AC1 = 0.894). The deposit ships six relational tables (bibliographic metadata, sentence-level annotations, named-entity rollups, article-level aggregates, per-category reliability tiers, and 9,462,845 BAAI/bge-m3 sentence-and-title embeddings). Raw newspaper text is excluded for copyright reasons; bibliographic coordinates (media, date, title, author, page_number) are sufficient for any researcher with institutional access to Factiva, Eureka.cc or ProQuest Canadian Newsstream to recover the original sentences. This deposit accompanies a methodology paper currently under revision at Scientific Data (Nature Portfolio).</p><p><strong>This deposit is the Apache Parquet mirror</strong> of the canonical PostgreSQL edition (sister deposit on Zenodo — DOI <a href="https://doi.org/10.5281/zenodo.20346364">10.5281/zenodo.20346364</a>). Each of the six relational tables is provided as a standalone <code>.parquet</code> file with ZSTD compression; the 1024-dimensional BAAI/bge-m3 embedding column is materialised as a <code>list<float></code>, and JSONB entity arrays are serialised as UTF-8 JSON strings. The schemas are otherwise identical to the PostgreSQL edition.</p><p>The Parquet bundle is readable natively by <code>pandas</code>, <code>polars</code>, R/<code>arrow</code>, <code>DuckDB</code>, and Spark without any database backend:</p><pre>import pandas as pd agg = pd.read_parquet('CCF_article_aggregates.parquet') emb = pd.read_parquet('CCF_sentence_embeddings.parquet')</pre><p>The HNSW index that ships with the PostgreSQL edition is not transferable to Parquet; brute-force cosine similarity remains tractable on the embedding column (≈ 9.46 M × 1024 float16).</p><p>The full annotation pipeline, training data, manual-annotation JSONL, intercoder-reliability benchmark, methodology manuscript (LaTeX sources + PDF), and reproducibility scripts are included in this deposit as <code>ccf_code_and_paper.tar.gz</code>. The same materials are also available on the project's OSF companion deposit (<a href="https://doi.org/10.17605/OSF.IO/Q5W47">10.17605/OSF.IO/Q5W47</a>) and on the development mirror at <a href="https://github.com/antoinelemor/CCF-canadian-climate-framing">GitHub</a>.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_20346373 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | CCF Database (Apache Parquet mirror): column-oriented edition of the Canadian Climate Framing corpus (1978–2024) Lemor, Antoine Pillod, Alizée Taylor, Matthew Nadeau, Richard climate change media analysis natural language processing BERT CamemBERT BAAI/bge-m3 pgvector Canada newspaper corpus framing analysis machine learning bilingual corpus semantic search named entity recognition <p>The Canadian Climate Framing (CCF) Database is a comprehensive, machine-learning-annotated corpus of climate-change media coverage in Canada. It comprises 266,271 articles from 20 major Canadian newspapers (1978–2024) processed into 9,198,958 two-sentence analytical units (82.9 % English, 17.1 % French). Each unit is annotated across 65 hierarchical categories by 128 BERT and CamemBERT classifiers, with a macro F1 of 0.866 on a 1,000-sentence gold standard (blind-phase Cohen's κ = 0.596, Krippendorff's α = 0.698, Gwet's AC1 = 0.894). The deposit ships six relational tables (bibliographic metadata, sentence-level annotations, named-entity rollups, article-level aggregates, per-category reliability tiers, and 9,462,845 BAAI/bge-m3 sentence-and-title embeddings). Raw newspaper text is excluded for copyright reasons; bibliographic coordinates (media, date, title, author, page_number) are sufficient for any researcher with institutional access to Factiva, Eureka.cc or ProQuest Canadian Newsstream to recover the original sentences. This deposit accompanies a methodology paper currently under revision at Scientific Data (Nature Portfolio).</p><p><strong>This deposit is the Apache Parquet mirror</strong> of the canonical PostgreSQL edition (sister deposit on Zenodo — DOI <a href="https://doi.org/10.5281/zenodo.20346364">10.5281/zenodo.20346364</a>). Each of the six relational tables is provided as a standalone <code>.parquet</code> file with ZSTD compression; the 1024-dimensional BAAI/bge-m3 embedding column is materialised as a <code>list<float></code>, and JSONB entity arrays are serialised as UTF-8 JSON strings. The schemas are otherwise identical to the PostgreSQL edition.</p><p>The Parquet bundle is readable natively by <code>pandas</code>, <code>polars</code>, R/<code>arrow</code>, <code>DuckDB</code>, and Spark without any database backend:</p><pre>import pandas as pd agg = pd.read_parquet('CCF_article_aggregates.parquet') emb = pd.read_parquet('CCF_sentence_embeddings.parquet')</pre><p>The HNSW index that ships with the PostgreSQL edition is not transferable to Parquet; brute-force cosine similarity remains tractable on the embedding column (≈ 9.46 M × 1024 float16).</p><p>The full annotation pipeline, training data, manual-annotation JSONL, intercoder-reliability benchmark, methodology manuscript (LaTeX sources + PDF), and reproducibility scripts are included in this deposit as <code>ccf_code_and_paper.tar.gz</code>. The same materials are also available on the project's OSF companion deposit (<a href="https://doi.org/10.17605/OSF.IO/Q5W47">10.17605/OSF.IO/Q5W47</a>) and on the development mirror at <a href="https://github.com/antoinelemor/CCF-canadian-climate-framing">GitHub</a>.</p> |
| title | CCF Database (Apache Parquet mirror): column-oriented edition of the Canadian Climate Framing corpus (1978–2024) |
| topic | climate change media analysis natural language processing BERT CamemBERT BAAI/bge-m3 pgvector Canada newspaper corpus framing analysis machine learning bilingual corpus semantic search named entity recognition |
| url | https://doi.org/10.5281/zenodo.20346373 |