Stance Detection on Social Media with Fine-Tuned Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gül, İlker, Lebret, Rémi, Aberer, Karl
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Social and Information Networks
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866929318955843584
author	Gül, İlker Lebret, Rémi Aberer, Karl
author_facet	Gül, İlker Lebret, Rémi Aberer, Karl
contents	Stance detection, a key task in natural language processing, determines an author's viewpoint based on textual analysis. This study evaluates the evolution of stance detection methods, transitioning from early machine learning approaches to the groundbreaking BERT model, and eventually to modern Large Language Models (LLMs) such as ChatGPT, LLaMa-2, and Mistral-7B. While ChatGPT's closed-source nature and associated costs present challenges, the open-source models like LLaMa-2 and Mistral-7B offers an encouraging alternative. Initially, our research focused on fine-tuning ChatGPT, LLaMa-2, and Mistral-7B using several publicly available datasets. Subsequently, to provide a comprehensive comparison, we assess the performance of these models in zero-shot and few-shot learning scenarios. The results underscore the exceptional ability of LLMs in accurately detecting stance, with all tested models surpassing existing benchmarks. Notably, LLaMa-2 and Mistral-7B demonstrate remarkable efficiency and potential for stance detection, despite their smaller sizes compared to ChatGPT. This study emphasizes the potential of LLMs in stance detection and calls for more extensive research in this field.
format	Preprint
id	arxiv_https___arxiv_org_abs_2404_12171
institution	arXiv
publishDate	2024
record_format	arxiv
spellingShingle	Stance Detection on Social Media with Fine-Tuned Large Language Models Gül, İlker Lebret, Rémi Aberer, Karl Computation and Language Social and Information Networks Stance detection, a key task in natural language processing, determines an author's viewpoint based on textual analysis. This study evaluates the evolution of stance detection methods, transitioning from early machine learning approaches to the groundbreaking BERT model, and eventually to modern Large Language Models (LLMs) such as ChatGPT, LLaMa-2, and Mistral-7B. While ChatGPT's closed-source nature and associated costs present challenges, the open-source models like LLaMa-2 and Mistral-7B offers an encouraging alternative. Initially, our research focused on fine-tuning ChatGPT, LLaMa-2, and Mistral-7B using several publicly available datasets. Subsequently, to provide a comprehensive comparison, we assess the performance of these models in zero-shot and few-shot learning scenarios. The results underscore the exceptional ability of LLMs in accurately detecting stance, with all tested models surpassing existing benchmarks. Notably, LLaMa-2 and Mistral-7B demonstrate remarkable efficiency and potential for stance detection, despite their smaller sizes compared to ChatGPT. This study emphasizes the potential of LLMs in stance detection and calls for more extensive research in this field.
title	Stance Detection on Social Media with Fine-Tuned Large Language Models
topic	Computation and Language Social and Information Networks
url	https://arxiv.org/abs/2404.12171