Staff View: :: Library Catalog

Saved in:

Bibliographic Details
Main Authors:	Orama, Jonathan Ayebakuro, Juvillà Garcia, Marc, Melero, Maite
Format:	Recurso digital
Language:
Published:	Zenodo 2025
Online Access:	https://doi.org/10.5281/zenodo.17950332
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866902240455819264
author	Orama, Jonathan Ayebakuro Juvillà Garcia, Marc Melero, Maite
author_facet	Orama, Jonathan Ayebakuro Juvillà Garcia, Marc Melero, Maite
contents	<div> <div>Dataset Summary</div> <br> <div>The CREXWET dataset is a multilingual corpus of Tweets annotated according to their relevance to a flash flood, or wildfire, for weather event detection. The dataset contains 627k (266k German,  223k English, 91k Spanish, 46k Catalan) unique sentences from various past flood and wildfire incidents, and some from unrelated time periods.</div> <br> <div>Supported Tasks and Leaderboards</div> <br> <div><code>text-classification</code>: This dataset can be used to train models for flood and wildfire detection.</div> <br> <div>Languages</div> <br> <div>The languages included in the dataset are:</div> <div>- English (`en`)</div> <div>- German (`de`)</div> <div>- Spanish (`es`)</div> <div>- Catalan (`ca-ES`)</div> <br> <h3>Dataset Structure</h3> <br> <div>Data Instances</div> <div> </div> <div>Each instance in the dataset contains the following fields:</div> <br> <div>- <code>id</code>: index id specific to this dataset.</div> <div>- <code>tweet_id</code>: id provided by Twitter/X.</div> <div>- <code>language</code>: language in sentence. Possible values: <code>ENGLISH, GERMAN, SPANISH, CATALAN</code>.</div> <div>- <code>label</code>: event label. Posible values: <code>fire, flood, none</code>.</div> <div>- <code>label_quality</code>: label quality score provided by <a href="https://github.com/cleanlab/cleanlab">Cleanlab</a>.</div> <br> <div> </div> <div>Data Fields</div> <br> <div><code>{</code></div> <div><code>  "id": "0",</code></div> <div><code>  "tweet_id": "276989327442079745",</code></div> <div><code>  "language": "GERMAN",</code></div> <div><code>  "label": "none",</code></div> <div><code>  "label_quality": "0.9916292981992856"</code></div> <div><code>}</code></div> <br> <div>Data Splits</div> <br> <div>The dataset is split into `train` and `test` sentences, the `train` may be split to create `dev` sentences, but `test` should only be used for evaluation. The annotation of `train` sentences are machine generated, while the `test` are machine generated and then human reviewed. Crisis incidents in `train` are not in `test`. The statistics of the splits are:</div> </div> <div> </div> <div> <table style="border-collapse: collapse; width: 22.8733%;"><colgroup><col style="width: 44.1048%;"><col style="width: 55.8952%;"></colgroup> <tbody> <tr> <td><strong>Dataset</strong></td> <td><strong>Instances</strong></td> </tr> <tr> <td>Train</td> <td>592,704</td> </tr> <tr> <td>Test</td> <td>34,722</td> </tr> </tbody> </table> <br> <div> </div> <h3>Dataset Creation</h3> <br> <div>Curation Rationale</div> <br> <div>This dataset was created to train a weather emergency detection model within the <a href="https://crexdata.eu/">CREXDATA</a> Project (Grant Agreement No. 101092749).</div> <br> <div>Source Data</div> <br> <div>Initial Data Collection and Normalization</div> <br> <div>The data was collected from Twitter, focusing on days prior, during, and after past floods and wildfire events in Europe and USA. For German we focused on Germany and Austria, for English on USA, for Spanish and Catalan on Spain. We also sourced data from public weather crisis datasets including <a href="https://crisislex.org/data-collections.html#CrisisLexT6">CrisisLexT6</a>, <a href="https://crisislex.org/data-collections.html#CrisisLexT26">CrisisLexT26</a>, <a href="https://crisisnlp.qcri.org/crisismmd">CrisisMMD</a>, <a href="https://crisisnlp.qcri.org/lrec2016/lrec2016.html">CrisisNLP</a>, <a href="https://crisisnlp.qcri.org/humaid_dataset">HumAID</a>.  </div> <br> <div>Who are the source language producers?</div> <br> <div>The source language producers are users of Twitter.</div> <br> <div>Annotations</div> <br> <div>Annotation process</div> <br> <div>The `train` dataset was annotated using <a href="https://huggingface.co/microsoft/Phi-3.5-mini-instruct">Phi 3.5 mini</a> a large language model from Microsoft. The LLM was provided the prompt below, and the text to be annotated was provided in place of `<TEXT>` in the prompt:</div> <div> </div> <div><code>Analyze the given body of text and determine whether it contains information relevant to an ongoing fire disaster, flood disaster, or neither.</code></div> <div><code>The classification should be based on the presence of specific, actionable details or key indicators of relevance to such disasters.</code></div> <div><code>Examine the text for mentions of fire-related or flood-related events, such as locations, impacts, safety advisories, or emergency responses.</code></div> <div><code>Classify as "fire" if the text contains information about wildfires, house fires, smoke hazards, evacuation orders due to fire, or related topics.</code></div> <div><code>Classify as "flood" if the text discusses water inundation, flash floods, storm surges, flood warnings, or evacuation due to flooding.</code></div> <div><code>Classify as "none" if the text does not provide information relevant to either disaster or lacks actionable disaster-related content.</code></div> <div><code>Texts that express sympathy, offer prayers, or request donations should also be classified as "none".</code></div> <div><code>Only assign "flood" or "fire" if the relevance is clear; otherwise, default to "none".</code></div> <div><code>The response should be formatted as follows:</code></div> <div><code>{"classification": "[flood/fire/none]"}</code></div> <div><code>Example</code></div> <div><code>Input: "Evacuation orders have been issued for Riverside due to rapidly spreading wildfire near the canyon."</code></div> <div><code>Output: {"classification": "fire"}</code></div> <div><code>Classify:</code></div> <div><code>Input: "<TEXT>"</code></div> <div> </div> <div>Once labelled, machined-enabled label cleaning is performed using <a href="https://github.com/cleanlab/cleanlab">Cleanlab</a> following their <a href="https://docs.cleanlab.ai/stable/index.html">documentation</a>. Error labels and low quality labels below 0.3 were dropped, we further re-labelled instances of `fire, flood` with label quality less than 0.7 as `none`. Our annotation and label cleaning process can be found at the <a href="https://github.com/altairengineering/crexdata-public/tree/main/WP4/Text-Mining/LLM-Auto-Annotation-Tool">repository</a>.</div> <br> <div>The `test` data was first machine annotated, cleaned, and then reviewed 2 human annotators. In this case, error labels are also reviewed and not automatically discarded.</div> <br> <div>Who are the annotators?</div> <br> <div>The annotators of the `test` have native or high-level profiency in English, Catalan, and Spanish.</div> <br> <div>Personal and Sensitive Information</div> <br> <div>N/A</div> <br> <div>Considerations for Using the Data</div> <br> <div>Social Impact of Dataset</div> <br> <div>We hope this data can improve research into weather emergency detection in social media data.</div> <br> <div>Discussion of Biases</div> <br> <div>We are aware that, since the data comes from social media, this will contain biases, hate speech and toxic content. We have not applied any steps to reduce their impact.</div> <br> <div>Other Known Limitations</div> <br> <div>The texts in the dataset have to be downloaded from Twitter, therefore some instances might be lost in cases where tweets have been deleted and are no longer available.</div> <br> <div>Additional Information</div> <br> <div>Dataset Curators</div> <br> <div>Language Technologies Unit (LangTech) at the Barcelona Supercomputing Center.</div> <br> <div>This work has been developed under the EU-funded <a href="https://crexdata.eu/">CREXDATA</a> Project (Grant Agreement No. 101092749).</div> <br> <div>Licensing Information</div> <br> <div><a href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0</a>.</div> </div>
format	Recurso digital
id	zenodo_https___doi_org_10_5281_zenodo_17950332
institution	Zenodo
language
publishDate	2025
publisher	Zenodo
record_format	zenodo
spellingShingle	CREXWET: CREdata-Weather-Emergency-Twitter Orama, Jonathan Ayebakuro Juvillà Garcia, Marc Melero, Maite <div> <div>Dataset Summary</div> <br> <div>The CREXWET dataset is a multilingual corpus of Tweets annotated according to their relevance to a flash flood, or wildfire, for weather event detection. The dataset contains 627k (266k German,  223k English, 91k Spanish, 46k Catalan) unique sentences from various past flood and wildfire incidents, and some from unrelated time periods.</div> <br> <div>Supported Tasks and Leaderboards</div> <br> <div><code>text-classification</code>: This dataset can be used to train models for flood and wildfire detection.</div> <br> <div>Languages</div> <br> <div>The languages included in the dataset are:</div> <div>- English (`en`)</div> <div>- German (`de`)</div> <div>- Spanish (`es`)</div> <div>- Catalan (`ca-ES`)</div> <br> <h3>Dataset Structure</h3> <br> <div>Data Instances</div> <div> </div> <div>Each instance in the dataset contains the following fields:</div> <br> <div>- <code>id</code>: index id specific to this dataset.</div> <div>- <code>tweet_id</code>: id provided by Twitter/X.</div> <div>- <code>language</code>: language in sentence. Possible values: <code>ENGLISH, GERMAN, SPANISH, CATALAN</code>.</div> <div>- <code>label</code>: event label. Posible values: <code>fire, flood, none</code>.</div> <div>- <code>label_quality</code>: label quality score provided by <a href="https://github.com/cleanlab/cleanlab">Cleanlab</a>.</div> <br> <div> </div> <div>Data Fields</div> <br> <div><code>{</code></div> <div><code>  "id": "0",</code></div> <div><code>  "tweet_id": "276989327442079745",</code></div> <div><code>  "language": "GERMAN",</code></div> <div><code>  "label": "none",</code></div> <div><code>  "label_quality": "0.9916292981992856"</code></div> <div><code>}</code></div> <br> <div>Data Splits</div> <br> <div>The dataset is split into `train` and `test` sentences, the `train` may be split to create `dev` sentences, but `test` should only be used for evaluation. The annotation of `train` sentences are machine generated, while the `test` are machine generated and then human reviewed. Crisis incidents in `train` are not in `test`. The statistics of the splits are:</div> </div> <div> </div> <div> <table style="border-collapse: collapse; width: 22.8733%;"><colgroup><col style="width: 44.1048%;"><col style="width: 55.8952%;"></colgroup> <tbody> <tr> <td><strong>Dataset</strong></td> <td><strong>Instances</strong></td> </tr> <tr> <td>Train</td> <td>592,704</td> </tr> <tr> <td>Test</td> <td>34,722</td> </tr> </tbody> </table> <br> <div> </div> <h3>Dataset Creation</h3> <br> <div>Curation Rationale</div> <br> <div>This dataset was created to train a weather emergency detection model within the <a href="https://crexdata.eu/">CREXDATA</a> Project (Grant Agreement No. 101092749).</div> <br> <div>Source Data</div> <br> <div>Initial Data Collection and Normalization</div> <br> <div>The data was collected from Twitter, focusing on days prior, during, and after past floods and wildfire events in Europe and USA. For German we focused on Germany and Austria, for English on USA, for Spanish and Catalan on Spain. We also sourced data from public weather crisis datasets including <a href="https://crisislex.org/data-collections.html#CrisisLexT6">CrisisLexT6</a>, <a href="https://crisislex.org/data-collections.html#CrisisLexT26">CrisisLexT26</a>, <a href="https://crisisnlp.qcri.org/crisismmd">CrisisMMD</a>, <a href="https://crisisnlp.qcri.org/lrec2016/lrec2016.html">CrisisNLP</a>, <a href="https://crisisnlp.qcri.org/humaid_dataset">HumAID</a>.  </div> <br> <div>Who are the source language producers?</div> <br> <div>The source language producers are users of Twitter.</div> <br> <div>Annotations</div> <br> <div>Annotation process</div> <br> <div>The `train` dataset was annotated using <a href="https://huggingface.co/microsoft/Phi-3.5-mini-instruct">Phi 3.5 mini</a> a large language model from Microsoft. The LLM was provided the prompt below, and the text to be annotated was provided in place of `<TEXT>` in the prompt:</div> <div> </div> <div><code>Analyze the given body of text and determine whether it contains information relevant to an ongoing fire disaster, flood disaster, or neither.</code></div> <div><code>The classification should be based on the presence of specific, actionable details or key indicators of relevance to such disasters.</code></div> <div><code>Examine the text for mentions of fire-related or flood-related events, such as locations, impacts, safety advisories, or emergency responses.</code></div> <div><code>Classify as "fire" if the text contains information about wildfires, house fires, smoke hazards, evacuation orders due to fire, or related topics.</code></div> <div><code>Classify as "flood" if the text discusses water inundation, flash floods, storm surges, flood warnings, or evacuation due to flooding.</code></div> <div><code>Classify as "none" if the text does not provide information relevant to either disaster or lacks actionable disaster-related content.</code></div> <div><code>Texts that express sympathy, offer prayers, or request donations should also be classified as "none".</code></div> <div><code>Only assign "flood" or "fire" if the relevance is clear; otherwise, default to "none".</code></div> <div><code>The response should be formatted as follows:</code></div> <div><code>{"classification": "[flood/fire/none]"}</code></div> <div><code>Example</code></div> <div><code>Input: "Evacuation orders have been issued for Riverside due to rapidly spreading wildfire near the canyon."</code></div> <div><code>Output: {"classification": "fire"}</code></div> <div><code>Classify:</code></div> <div><code>Input: "<TEXT>"</code></div> <div> </div> <div>Once labelled, machined-enabled label cleaning is performed using <a href="https://github.com/cleanlab/cleanlab">Cleanlab</a> following their <a href="https://docs.cleanlab.ai/stable/index.html">documentation</a>. Error labels and low quality labels below 0.3 were dropped, we further re-labelled instances of `fire, flood` with label quality less than 0.7 as `none`. Our annotation and label cleaning process can be found at the <a href="https://github.com/altairengineering/crexdata-public/tree/main/WP4/Text-Mining/LLM-Auto-Annotation-Tool">repository</a>.</div> <br> <div>The `test` data was first machine annotated, cleaned, and then reviewed 2 human annotators. In this case, error labels are also reviewed and not automatically discarded.</div> <br> <div>Who are the annotators?</div> <br> <div>The annotators of the `test` have native or high-level profiency in English, Catalan, and Spanish.</div> <br> <div>Personal and Sensitive Information</div> <br> <div>N/A</div> <br> <div>Considerations for Using the Data</div> <br> <div>Social Impact of Dataset</div> <br> <div>We hope this data can improve research into weather emergency detection in social media data.</div> <br> <div>Discussion of Biases</div> <br> <div>We are aware that, since the data comes from social media, this will contain biases, hate speech and toxic content. We have not applied any steps to reduce their impact.</div> <br> <div>Other Known Limitations</div> <br> <div>The texts in the dataset have to be downloaded from Twitter, therefore some instances might be lost in cases where tweets have been deleted and are no longer available.</div> <br> <div>Additional Information</div> <br> <div>Dataset Curators</div> <br> <div>Language Technologies Unit (LangTech) at the Barcelona Supercomputing Center.</div> <br> <div>This work has been developed under the EU-funded <a href="https://crexdata.eu/">CREXDATA</a> Project (Grant Agreement No. 101092749).</div> <br> <div>Licensing Information</div> <br> <div><a href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0</a>.</div> </div>
title	CREXWET: CREdata-Weather-Emergency-Twitter
url	https://doi.org/10.5281/zenodo.17950332

Key	Value
ui	"standard"
VUFIND_SESSION	"4c21694578ede28e5d81c9cc690b928e"
language	"en"

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1784672557.6802 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "4c21694578ede28e5d81c9cc690b928e" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1578}
SessionHelper	Laminas\Stdlib\ArrayObject {#1773}

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"ui=standard; VUFIND_SESSION=4c21694578ede28e5d81c9cc690b928e; language=en"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.216.85"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"54041"
REDIRECT_URL	"/Record/zenodo_https___doi_org_10_5281_zenodo_17950332/Details"
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/zenodo_https___doi_org_10_5281_zenodo_17950332/Details"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1784672557.6758
REQUEST_TIME	1784672557

Key	Value
ui	"standard"
VUFIND_SESSION	"4c21694578ede28e5d81c9cc690b928e"
language	"en"

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1784672557.6802 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "4c21694578ede28e5d81c9cc690b928e" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1578}
SessionHelper	Laminas\Stdlib\ArrayObject {#1773}

Environment & details:

Environment & details:

Similar Items

Environment & details:

Environment & details: