FarExStance: Explainable Stance Detection for Farsi
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915069836656640 |
|---|---|
| author | Zarharan, Majid Hashemi, Maryam Behroozrazegh, Malika Eetemadi, Sauleh Pilehvar, Mohammad Taher Foster, Jennifer |
| author_facet | Zarharan, Majid Hashemi, Maryam Behroozrazegh, Malika Eetemadi, Sauleh Pilehvar, Mohammad Taher Foster, Jennifer |
| contents | We introduce FarExStance, a new dataset for explainable stance detection in Farsi. Each instance in this dataset contains a claim, the stance of an article or social media post towards that claim, and an extractive explanation which provides evidence for the stance label. We compare the performance of a fine-tuned multilingual RoBERTa model to several large language models in zero-shot, few-shot, and parameter-efficient fine-tuned settings on our new dataset. On stance detection, the most accurate models are the fine-tuned RoBERTa model, the LLM Aya-23-8B which has been fine-tuned using parameter-efficient fine-tuning, and few-shot Claude-3.5-Sonnet. Regarding the quality of the explanations, our automatic evaluation metrics indicate that few-shot GPT-4o generates the most coherent explanations, while our human evaluation reveals that the best Overall Explanation Score (OES) belongs to few-shot Claude-3.5-Sonnet. The fine-tuned Aya-32-8B model produced explanations most closely aligned with the reference explanations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_14008 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FarExStance: Explainable Stance Detection for Farsi Zarharan, Majid Hashemi, Maryam Behroozrazegh, Malika Eetemadi, Sauleh Pilehvar, Mohammad Taher Foster, Jennifer Computation and Language We introduce FarExStance, a new dataset for explainable stance detection in Farsi. Each instance in this dataset contains a claim, the stance of an article or social media post towards that claim, and an extractive explanation which provides evidence for the stance label. We compare the performance of a fine-tuned multilingual RoBERTa model to several large language models in zero-shot, few-shot, and parameter-efficient fine-tuned settings on our new dataset. On stance detection, the most accurate models are the fine-tuned RoBERTa model, the LLM Aya-23-8B which has been fine-tuned using parameter-efficient fine-tuning, and few-shot Claude-3.5-Sonnet. Regarding the quality of the explanations, our automatic evaluation metrics indicate that few-shot GPT-4o generates the most coherent explanations, while our human evaluation reveals that the best Overall Explanation Score (OES) belongs to few-shot Claude-3.5-Sonnet. The fine-tuned Aya-32-8B model produced explanations most closely aligned with the reference explanations. |
| title | FarExStance: Explainable Stance Detection for Farsi |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.14008 |